修改dtypes.py清除tensflow日志

这篇博客介绍了如何修改TensorFlow的日志路径中`dtypes.py`文件里的量子整数类型定义。原始代码使用了一维数组,而更新后的代码将每个量子整数类型定义为固定大小的一维数组,例如:_np_qint8更改为_np_qint8=np.dtype([qint8 np.int8, (1,)])。这一改动可能与优化内存使用或兼容性有关。
部署运行你感兴趣的模型镜像
举例: 
_np_qint32 C:\Users\Administrator\AppData\Local\Programs\Python
\Python36\lib\site-packages\tensorflow\python\framework
\dtypes.py

看tensflow日志上显示的日志路径中dtypes.py
依次修改

举例:
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
依次改为: 
_np_qint8 = np.dtype([("qint8", np.int8, (1,))])
_np_quint8 = np.dtype([("quint8", np.uint8, (1,))])
_np_qint16 = np.dtype([("qint16", np.int16, (1,))])
_np_quint16 = np.dtype([("quint16", np.uint16, (1,))])
_np_qint32 = np.dtype([("qint32", np.int32, (1,))])

您可能感兴趣的与本文相关的镜像

TensorFlow-v2.15

TensorFlow-v2.15

TensorFlow

TensorFlow 是由Google Brain 团队开发的开源机器学习框架,广泛应用于深度学习研究和生产环境。 它提供了一个灵活的平台,用于构建和训练各种机器学习模型

D:\anaconda3\envs\tf_legacy\lib\site-packages\tensorflow\python\framework\dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) D:\anaconda3\envs\tf_legacy\lib\site-packages\tensorflow\python\framework\dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) D:\anaconda3\envs\tf_legacy\lib\site-packages\tensorflow\python\framework\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) D:\anaconda3\envs\tf_legacy\lib\site-packages\tensorflow\python\framework\dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) D:\anaconda3\envs\tf_legacy\lib\site-packages\tensorflow\python\framework\dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) D:\anaconda3\envs\tf_legacy\lib\site-packages\tensorflow\python\framework\dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) usage: freeze_graph.py [-h] -bert_model_dir BERT_MODEL_DIR -model_dir MODEL_DIR [-model_pb_dir MODEL_PB_DIR] [-max_seq_len MAX_SEQ_LEN] [-n
03-27
>>> import tensorflow as tf A module that was compiled using NumPy 1.x cannot be run in NumPy 2.2.6 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'. If you are a user of the module, the easiest solution will be to downgrade to 'numpy<2' or try to upgrade the affected module. We expect that some modules will need time to support NumPy 2. Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\__init__.py", line 37, in <module> from tensorflow.python.tools import module_util as _module_util File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\__init__.py", line 37, in <module> from tensorflow.python.eager import context File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\eager\context.py", line 35, in <module> from tensorflow.python.client import pywrap_tf_session File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\client\pywrap_tf_session.py", line 19, in <module> from tensorflow.python.client._pywrap_tf_session import * AttributeError: _ARRAY_API not found A module that was compiled using NumPy 1.x cannot be run in NumPy 2.2.6 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'. If you are a user of the module, the easiest solution will be to downgrade to 'numpy<2' or try to upgrade the affected module. We expect that some modules will need time to support NumPy 2. Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\__init__.py", line 37, in <module> from tensorflow.python.tools import module_util as _module_util File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\__init__.py", line 42, in <module> from tensorflow.python import data File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\__init__.py", line 21, in <module> from tensorflow.python.data import experimental File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\experimental\__init__.py", line 96, in <module> from tensorflow.python.data.experimental import service File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\experimental\service\__init__.py", line 419, in <module> from tensorflow.python.data.experimental.ops.data_service_ops import distribute File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\experimental\ops\data_service_ops.py", line 24, in <module> from tensorflow.python.data.experimental.ops import compression_ops File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\experimental\ops\compression_ops.py", line 16, in <module> from tensorflow.python.data.util import structure File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\util\structure.py", line 23, in <module> from tensorflow.python.data.util import nest File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\util\nest.py", line 36, in <module> from tensorflow.python.framework import sparse_tensor as _sparse_tensor File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\framework\sparse_tensor.py", line 24, in <module> from tensorflow.python.framework import constant_op File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\framework\constant_op.py", line 25, in <module> from tensorflow.python.eager import execute File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\eager\execute.py", line 23, in <module> from tensorflow.python.framework import dtypes File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\framework\dtypes.py", line 29, in <module> from tensorflow.python.lib.core import _pywrap_bfloat16 AttributeError: _ARRAY_API not found ImportError: numpy.core._multiarray_umath failed to import ImportError: numpy.core.umath failed to import Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\__init__.py", line 37, in <module> from tensorflow.python.tools import module_util as _module_util File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\__init__.py", line 42, in <module> from tensorflow.python import data File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\__init__.py", line 21, in <module> from tensorflow.python.data import experimental File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\experimental\__init__.py", line 96, in <module> from tensorflow.python.data.experimental import service File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\experimental\service\__init__.py", line 419, in <module> from tensorflow.python.data.experimental.ops.data_service_ops import distribute File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\experimental\ops\data_service_ops.py", line 24, in <module> from tensorflow.python.data.experimental.ops import compression_ops File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\experimental\ops\compression_ops.py", line 16, in <module> from tensorflow.python.data.util import structure File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\util\structure.py", line 23, in <module> from tensorflow.python.data.util import nest File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\data\util\nest.py", line 36, in <module> from tensorflow.python.framework import sparse_tensor as _sparse_tensor File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\framework\sparse_tensor.py", line 24, in <module> from tensorflow.python.framework import constant_op File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\framework\constant_op.py", line 25, in <module> from tensorflow.python.eager import execute File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\eager\execute.py", line 23, in <module> from tensorflow.python.framework import dtypes File "C:\Users\SHINING3D\anaconda3\envs\gpu_env\lib\site-packages\tensorflow\python\framework\dtypes.py", line 34, in <module> _np_bfloat16 = _pywrap_bfloat16.TF_bfloat16_type() TypeError: Unable to convert function return value to a Python type! The signature was () -> handle >>> print("TF Version:", tf.__version__) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'tf' is not defined >>> print("GPU Available:", tf.config.list_physical_devices('GPU')) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'tf' is not defined
最新发布
10-15
<think>嗯,用户之前遇到了依赖冲突的问题,现在按照我的建议安装了指定版本的包,但出现了新的错误:`load_tensorflow`的跟踪栈,特别是关于`ValueError: Failed to import metagraph`。我需要分析可能的原因,并提供解决方案。 首先,用户的问题出现在加载TensorFlow模型到RKNN Toolkit时。错误信息提到了`tf_optimizer.py`中的失败,这可能与TensorFlow版本或模型结构有关。用户已经安装了指定的TensorFlow 2.6.2,但可能还有其他因素导致这个问题。 可能的因素包括: 1. **模型格式问题**:用户可能使用了SavedModel格式,而RKNN Toolkit需要冻结的GraphDef(.pb)模型。 2. **模型中的自定义层或操作**:如果模型包含RKNN不支持的层或操作,转换时会失败。 3. **TensorFlow安装问题**:虽然版本正确,但可能存在安装不完整或依赖项冲突。 4. **Python环境问题**:虚拟环境中可能存在残留文件或其他包的干扰。 接下来,我需要分步骤思考解决方案: **步骤一:检查模型格式** 用户可能尝试加载的是SavedModel,而RKNN可能需要冻结的.pb文件。建议用户使用`freeze_graph`工具将模型转换为冻结格式。同时,确保输入节点名称正确,可能需要使用TensorFlow的工具来查看模型结构。 **步骤二:验证模型兼容性** 用户需要检查模型中是否有自定义层或RKNN不支持的操作。运行模型在纯TensorFlow环境中的推理,确保模型本身没有问题。另外,可以尝试简化模型结构,排除复杂操作的影响。 **步骤三:环境彻底清理** 虽然用户创建了虚拟环境,但可能存在安装时的缓存或残留。建议彻底卸载TensorFlow和相关包,然后重新安装,确保所有依赖项都正确。 **步骤四:使用中间转换格式** 如果直接转换TensorFlow模型失败,可以尝试先将模型转换为ONNX格式,再通过RKNN加载ONNX模型。这可能需要额外的工具,如tf2onnx,但可能绕过一些兼容性问题。 **步骤五:日志和调试** 启用详细日志有助于定位问题。查看是否有更具体的错误信息,或者尝试在转换时设置不同的日志级别。此外,检查是否有路径或文件权限问题,导致模型无法正确读取。 **步骤六:寻求官方支持** 如果以上步骤都失败,建议用户查阅RKNN的官方文档或社区,看看是否有已知的解决方案。可能存在特定版本的RKNN Toolkit需要额外的补丁或配置。 最后,考虑到用户之前已经处理过依赖冲突,这次的问题可能更具体于模型本身或环境配置。需要逐步排除可能的原因,从模型格式到环境设置,确保每一步都符合RKNN Toolkit的要求。</think>━━━━━━━━━━━━━━━━━━ ### 错误根源分析 该错误发生在使用RKNN Toolkit加载TensorFlow模型时,核心原因是 **TensorFlow模型结构解析失败**。常见触发场景: ``` 1. 使用了不支持的模型格式(如SavedModel而非Frozen Graph) 2. 模型中包含RKNN不支持的TensorFlow操作 3. TensorFlow版本与模型结构不兼容(即使版本号匹配) ``` ━━━━━━━━━━━━━━━━━━ ### 逐步解决方案 #### 步骤一:确认模型格式要求 **必须使用冻结的TensorFlow GraphDef模型(.pb文件)** ```bash # 检查模型格式是否正确 file your_model.pb # 期望输出:data (protocol buffer binary format) # 查看模型输入输出节点名称 saved_model_cli show --dir your_model/ --all ``` > 📌 如果使用SavedModel格式,需先转换: > ```python > from tensorflow.python.tools import freeze_graph > freeze_graph.freeze_graph(...) # 参考官方冻结模型文档 > ``` #### 步骤二:强制指定输入输出节点 在`load_tensorflow`调用中显式声明节点名称: ```python rknn.load_tensorflow( tf_pb='model.pb', inputs=['input_1'], # 必须精确匹配输入节点名 outputs=['output_1'], # 必须精确匹配输出节点名 input_size_list=[[224,224,3]] ) ``` #### 步骤三:检查模型操作兼容性 运行以下代码验证模型是否包含非法操作: ```python import tensorflow as tf # 加载模型 with tf.io.gfile.GFile('model.pb', 'rb') as f: graph_def = tf.compat.v1.GraphDef() graph_def.ParseFromString(f.read()) # 打印所有操作类型 ops = {node.op for node in graph_def.node} print(f"模型中包含的操作类型:{ops}") # 对比RKNN支持的操作列表 # 参考:https://github.com/rockchip-linux/rknn-toolkit2/blob/master/doc/RKNN_OP_List.md ``` > ⚠️ 若出现`TensorList*`、`While`等控制流操作,需修改模型结构 #### 步骤四:环境完整性验证 在虚拟环境中执行以下检查: ```bash # 确认TensorFlow精确版本 python -c "import tensorflow as tf; print(tf.__version__)" # 必须输出2.6.2 # 清理残留安装 pip uninstall -y tensorflow protobuf numpy pip install --no-cache-dir tensorflow==2.6.2 protobuf==3.12.2 # 检查ABI兼容性 python -c "import tensorflow as tf; print(tf.sysconfig.get_compile_flags())" # 应包含 '-D_GLIBCXX_USE_CXX11_ABI=0' ``` #### 步骤五:使用调试模式获取详细信息 ```python from rknn.api import RKNN rknn = RKNN(verbose=True) # 启用详细日志 rknn.config(target_platform='rk3588') # 根据实际硬件修改 ret = rknn.load_tensorflow(...) # 观察完整错误堆栈 ``` > 📍 关键日志线索: > - 若出现`Unsupported Ops: [...]`需修改模型 > - 若出现`Failed to parse meta graph`需检查模型冻结方式 ━━━━━━━━━━━━━━━━━━ ### 高级调试方案 #### 方案一:强制优化器降级 编辑RKNN源码(谨慎操作): ```python # 定位文件:rknn/base/convertor/tensorflow2onnx/tf_loader.py # 修改第722行附近的优化参数 config = tf.compat.v1.ConfigProto() config.graph_options.rewrite_options.disable_meta_optimizer = True # 新增 ``` #### 方案二:转ONNX中间格式 ```bash # 使用tf2onnx转换模型 python -m tf2onnx.convert --saved-model saved_model_dir --output model.onnx # 在RKNN中加载ONNX rknn.load_onnx(model='model.onnx') ``` #### 方案三:使用官方Docker环境 ```bash # 拉取Rockchip官方镜像 docker pull rockchip/rknn-toolkit2:1.5.0 # 运行容器并挂载模型目录 docker run -it --rm -v /path/to/models:/models rockchip/rknn-toolkit2:1.5.0 ``` ━━━━━━━━━━━━━━━━━━ ### 关键排查流程图 ``` 模型加载失败 ├─→ 检查模型格式 → 非冻结模型 → 使用freeze_graph转换 ├─→ 检查节点名称 → 名称错误 → 使用Netron可视化模型 ├─→ 检查操作兼容性 → 存在非法操作 → 修改模型结构 └─→ 检查TF版本 → 版本异常 → 完全重装指定版本环境 ``` 若仍无法解决,建议: 1. 提供完整的模型结构图和转换日志 2. 测试Rockchip提供的预训练模型是否可转换(验证环境完整性) 3. 在RK3588开发板上直接执行转换(部分操作需硬件支持)
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值