Compile TensorFlow with CMake

本文档提供了详细的步骤说明,指导如何将TensorFlow与CMake项目无缝集成,包括安装TensorFlow、配置编译环境、安装依赖库Eigen和Protobuf等关键环节。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

https://github.com/cjweeks/tensorflow-cmake


update cmake:

wget http://www.cmake.org/files/v3.4/cmake-3.4.1.tar.gz

tar -xvf cmake-3.4.1.tar.gz

./configure

make && make install

export  CMAKE_ROOT=$HOME/local/share/cmake-3.4

tensorflow-cmake

Integrate TensorFlow with CMake projects effortlessly.

TensorFlow

TensorFlow is an amazing tool for machine learning and intelligence using computational graphs.TensorFlow includes APIs for both Python and C++, although the C++ API is slightly less documented. However, the most standardway to integrate C++ projects with TensorFlow is to build the projectinside the TensorFlow repository, yielding a massive binary.Additionally,Bazel is the only certified way to build such projects. This document and the code in thisrepository will allow one to integrate TensorFlow with CMake projects without producing a large binary.

Note: The instructions here correspond to an Ubuntu Linux environment; although some commands may differ for other operating systems and distributions, the general ideas are identical.

Step 1: Install TensorFlow

Follow the instructions for installing Bazel. Install dependencies and cloneTensorFlow from its git repository:

sudo apt-get install autoconf automake libtool curl make g++ unzip  # Protobuf Dependencies
sudo apt-get install python-numpy swig python-dev python-wheel      # TensorFlow Dependencies
git clone https://github.com/tensorflow/tensorflow                  # TensorFlow

Enter the cloned repository, and append the following to the tensorflow/BUILD file:

# Added build rule
cc_binary(
    name = "libtensorflow_all.so",
    linkshared = 1,
    linkopts = ["-Wl,--version-script=tensorflow/tf_version_script.lds"],
    deps = [
        "//tensorflow/cc:cc_ops",
        "//tensorflow/core:framework_internal",
        "//tensorflow/core:tensorflow",
    ],
)

This specifies a new build rule, producing libtensorflow_all.so, that includes all the required dependencies for integrationwith a C++ project. Build the shared library and copy it to/usr/local/lib as follows:

./configure      # Note that this requires user input
bazel build tensorflow:libtensorflow_all.so
sudo cp bazel-bin/tensorflow/libtensorflow_all.so /usr/local/lib

Copy the source to /usr/local/include/google and remove unneeded items:

sudo mkdir -p /usr/local/include/google/tensorflow
sudo cp -r tensorflow /usr/local/include/google/tensorflow/
sudo find /usr/local/include/google/tensorflow/tensorflow -type f  ! -name "*.h" -delete

Copy all generated files from bazel-genfiles:

sudo cp bazel-genfiles/tensorflow/core/framework/*.h  /usr/local/include/google/tensorflow/tensorflow/core/framework
sudo cp bazel-genfiles/tensorflow/core/kernels/*.h  /usr/local/include/google/tensorflow/tensorflow/core/kernels
sudo cp bazel-genfiles/tensorflow/core/lib/core/*.h  /usr/local/include/google/tensorflow/tensorflow/core/lib/core
sudo cp bazel-genfiles/tensorflow/core/protobuf/*.h  /usr/local/include/google/tensorflow/tensorflow/core/protobuf
sudo cp bazel-genfiles/tensorflow/core/util/*.h  /usr/local/include/google/tensorflow/tensorflow/core/util
sudo cp bazel-genfiles/tensorflow/cc/ops/*.h  /usr/local/include/google/tensorflow/tensorflow/cc/ops

Copy the third party directory:

sudo cp -r third_party /usr/local/include/google/tensorflow/
sudo rm -r /usr/local/include/google/tensorflow/third_party/py
sudo rm -r /usr/local/include/google/tensorflow/third_party/avro

Step 2: Install Eigen and Protobuf

The TensorFlow runtime library requires both Protobuf and Eigen.However, specific versions are required, and these may clash with currently installed versions of either software. Therefore, two options areprovided:

  • Install the packages to a directory on your computer, which will overwrite / clash with any previous versions installed in that directory (but allow multiple projects to reference them).The default directory is/usr/local, but any may be specified to avoid clashing. This is the recommended option.
  • Add the packages as external dependencies, allowing CMake to download and build them inside the project directory, not affecting any current versions. This will never result in clashing,but the build process of your project may be lengthened.

Choose the option that best fits your needs; you may mix these options as well, installing one to/usr/local, while keeping the other confined in the current project. In the following instructions, be sure to replace<EXECUTABLE_NAME> with the name of your executable. Additionally, all generated CMake files should generally be placed in your CMake modules directory, which is commonly<PROJECT_ROOT>/cmake/Modules.

Eigen: Installing Locally

Execute the eigen.sh script as follows: sudo eigen.sh install <tensorflow-root> [<install-dir> <download-dir>]. Theinstall command specifies that Eigen is to be installed to a directory. The<tensorflow-root> argument should be the root of the TensorFlow repository. The optional<install-dir> argument allows you to specify the installation directory;this defaults to/usr/local but may be changed to avoid other versions. The<download-dir argument specifies the directory where Eigen will be download and extracted; this defaultsto the current directory.

To generate the needed CMake files for your project, execute the script as follows:eigen.sh generate installed [<cmake-dir> <install-dir>]. Thegenerate command specifies that the required CMake files are to be generated and placed in<cmake-dir> (this defaults to the current directory, but generally should your CMake modules directory). The optional<install-dir>argument specifies the directory Protobuf is installed to. This defaults to/usr/local and should directly correspond to the install directory specified when installing above. Two fileswill be copied to the specified directory:FindEigen.cmake andEigen_VERSION.cmake. Add the following to yourCMakeLists.txt:

# Eigen
find_package(Eigen REQUIRED)
include_directories(${Eigen_INCLUDE_DIRS})

Eigen: Adding as External Dependency

Execute the eigen.sh script as follows: eigen.sh generate external <tensorflow-root> [<cmake-dir>]. Theexternal command specifies that Eigen is notinstalled, but rather should be treated as an external CMake dependency. The<tensorflow-root> argument again should be the root directory of the TensorFlow repository,and the optional<cmake-dir> argument is the location to copy the required CMake modules to (defaults to the current directory). Two files will be copiedto the specified directory:Eigen.cmake and Eigen_VERSION.cmake. Add the following to yourCMakeLists.txt:

# Eigen
include(Eigen)
add_dependencies(<EXECUTABLE_NAME> Eigen)

Protobuf: Installing Locally

Execute the protobuf.sh script as follows: sudo protobuf.sh install <tensorflow-root> [<cmake-dir>]. The arguments are identical to those described in the Eigensection above.

Generate the required files as follows: protobuf.sh generate installed [<cmake-dir> <install-dir>]; the arguments are also identical to those above. Two files will be copied to the specified directory:FindProtobuf.cmake andProtobuf_VERSION.cmake. CMake provides us with aFindProtobuf.cmakemodule, but we will use our own, since we must specify the directory Protobuf was installed to. Add the following to yourCMakeLists.txt:

# Protobuf
find_package(Protobuf REQUIRED)
include_directories(${Protobuf_INCLUDE_DIRS})
target_link_libraries(<EXECUTABLE_NAME> ${Protobuf_LIBRARIES})

Protobuf: Adding as External Dependency

Execute the protobuf.sh script as follows: protobuf.sh generate external <tensorflow-root> [<cmake-dir>]. The arguments are also identical to those described in the Eigensection above. Two files will be copied to the specified directory:Protobuf.cmake and Protobuf_VERSION.cmake. Add the following to yourCMakeLists.txt:

# Protobuf
include(Protobuf)
add_dependencies(<EXECUTABLE_NAME> Protobuf)

Step 3: Configure the CMake Project

Edit your CMakeLists.txt to append your custom modules directory to the list of CMake modules (this is a common step in most CMake programs):

list(APPEND CMAKE_MODULE_PATH <CMAKE_MODULE_DIR>)
# Replace <CMAKE_MODULE_DIR> with your path
# The most common path is ${PROJECT_SOURCE_DIR}/cmake/Modules

If either Protobuf or Eigen was added as an external dependency, add the following to yourCMakeLists.txt:

# Specify download location
set (DOWNLOAD_LOCATION "${PROJECT_SOURCE_DIR}/external/src"
        CACHE PATH "Location where external projects will be downloaded")
mark_as_advanced(DOWNLOAD_LOCATION)

The projects in the examples/ directory demonstrate the correct usage of these instructions.

Troubleshooting

Compiler Path Build Error

If Bazel fails to build the TensorFlow library, stating error: Could not find compiler "gcc" in PATH, you may have to execute the following:

bazel clean                                   # Clean project
export CC="/usr/bin/gcc"                      # Set location of C compiler
export CXX="/usr/bin/g++"                     # Set location of C++ compiler
bazel build tensorflow:libtensorflow_all.so   # Rebuild project

Makefile with contrib/Makefile
1 tensorflow/core/platform/env.cc:304:9: error: conflicting return type specified for ‘virtual tensorflow::int64 tensorflow::{anonymous}::FileStream::ByteCount() const’
由于protobuf版本问题引起,需要尝试其它branch

eigen problem
download eigen from 
https://bitbucket.org/eigen/eigen/
tar -xvf and cpy to contrib/cmake_build/eigen/src/eigen

华为云Ascend C算子开发 环境搭(MindSpore) [ma-user SigmoidCustom]$bash build.sh using ASCEND_HOME_PATH: /home/ma-user/Ascend/ascend-toolkit/latest Preset CMake variables: ASCEND_COMPUTE_UNIT:STRING="ascend310b;ascend910b" ASCEND_PYTHON_EXECUTABLE:STRING="python3" CMAKE_BUILD_TYPE:STRING="Release" CMAKE_CROSS_PLATFORM_COMPILER:PATH="/usr/bin/aarch64-linux-gnu-g++" CMAKE_INSTALL_PREFIX:PATH="/home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out" ENABLE_BINARY_PACKAGE:BOOL="True" ENABLE_CROSS_COMPILE:BOOL="False" ENABLE_SOURCE_PACKAGE:BOOL="True" ENABLE_TEST:BOOL="True" vendor_name:STRING="customize" -- The C compiler identification is GNU 7.3.0 CMake Warning (dev) at /home/ma-user/work/cmake-3.28.3-linux-aarch64/share/cmake-3.28/Modules/CMakeFindBinUtils.cmake:224 (find_program): Policy CMP0109 is not set: find_program() requires permission to execute but not to read. Run "cmake --help-policy CMP0109" for policy details. Use the cmake_policy command to set the policy and suppress this warning. The file /usr/bin/ld is executable but not readable. CMake is ignoring it for compatibility. Call Stack (most recent call first): /home/ma-user/work/cmake-3.28.3-linux-aarch64/share/cmake-3.28/Modules/CMakeDetermineCCompiler.cmake:196 (include) CMakeLists.txt:2 (project) This warning is for project developers. Use -Wno-dev to suppress it. CMake Warning (dev) at /home/ma-user/work/cmake-3.28.3-linux-aarch64/share/cmake-3.28/Modules/CMakeFindBinUtils.cmake:224 (find_program): Policy CMP0109 is not set: find_program() requires permission to execute but not to read. Run "cmake --help-policy CMP0109" for policy details. Use the cmake_policy command to set the policy and suppress this warning. The file /bin/ld is executable but not readable. CMake is ignoring it for compatibility. Call Stack (most recent call first): /home/ma-user/work/cmake-3.28.3-linux-aarch64/share/cmake-3.28/Modules/CMakeDetermineCCompiler.cmake:196 (include) CMakeLists.txt:2 (project) This warning is for project developers. Use -Wno-dev to suppress it. -- The CXX compiler identification is GNU 7.3.0 CMake Warning (dev) at /home/ma-user/work/cmake-3.28.3-linux-aarch64/share/cmake-3.28/Modules/CMakeFindBinUtils.cmake:224 (find_program): Policy CMP0109 is not set: find_program() requires permission to execute but not to read. Run "cmake --help-policy CMP0109" for policy details. Use the cmake_policy command to set the policy and suppress this warning. The file /usr/bin/ld is executable but not readable. CMake is ignoring it for compatibility. Call Stack (most recent call first): /home/ma-user/work/cmake-3.28.3-linux-aarch64/share/cmake-3.28/Modules/CMakeDetermineCXXCompiler.cmake:202 (include) CMakeLists.txt:2 (project) This warning is for project developers. Use -Wno-dev to suppress it. CMake Warning (dev) at /home/ma-user/work/cmake-3.28.3-linux-aarch64/share/cmake-3.28/Modules/CMakeFindBinUtils.cmake:224 (find_program): Policy CMP0109 is not set: find_program() requires permission to execute but not to read. Run "cmake --help-policy CMP0109" for policy details. Use the cmake_policy command to set the policy and suppress this warning. The file /bin/ld is executable but not readable. CMake is ignoring it for compatibility. Call Stack (most recent call first): /home/ma-user/work/cmake-3.28.3-linux-aarch64/share/cmake-3.28/Modules/CMakeDetermineCXXCompiler.cmake:202 (include) CMakeLists.txt:2 (project) This warning is for project developers. Use -Wno-dev to suppress it. -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Opbuild generating sources -- Opbuild generating sources - done -- Configuring done (3.3s) -- Generating done (0.0s) CMake Warning: Manually-specified variables were not used by the project: CMAKE_CROSS_PLATFORM_COMPILER -- Build files have been written to: /home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out [ 7%] Building CXX object op_host/CMakeFiles/cust_optiling.dir/sigmoid_custom.cpp.o [ 14%] Building CXX object framework/tf_plugin/CMakeFiles/cust_tf_parsers.dir/tensorflow_sigmoid_custom_plugin.cc.o [ 21%] Generating scripts/install.sh, scripts/upgrade.sh [ 28%] Building CXX object op_host/CMakeFiles/cust_opapi.dir/__/autogen/aclnn_sigmoid_custom.cpp.o [ 28%] Built target gen_version_info [ 57%] Building CXX object op_host/CMakeFiles/cust_op_proto.dir/__/autogen/op_proto.cc.o [ 57%] Generating tbe/op_info_cfg/ai_core/ascend310b/aic-ascend310b-ops-info.json [ 57%] Generating tbe/.impl_timestamp [ 64%] Building CXX object op_host/CMakeFiles/cust_op_proto.dir/sigmoid_custom.cpp.o [ 64%] Generating tbe/op_info_cfg/ai_core/ascend910b/aic-ascend910b-ops-info.json [ 71%] Generating tbe/op_info_cfg/ai_core/npu_supported_ops.json /home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out/autogen /home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out/op_kernel/tbe/op_info_cfg/ai_core [ 71%] Built target modify_vendor ==============check valid for ops info start============== ==============check valid for ops info end================ ==============check valid for ops info start============== ==============check valid for ops info end================ Compile op info cfg successfully. Compile op info cfg successfully. [ 71%] Built target ops_info_gen_ascend310b [ 71%] Built target ops_info_gen_ascend910b [INFO] Succed generated /home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out/op_kernel/tbe/op_info_cfg/ai_core/npu_supported_ops.json [ 71%] Built target ascendc_impl_gen [ 71%] Built target npu_supported_ops [ 78%] Linking CXX shared library libcust_opapi.so [ 78%] Built target cust_opapi [ 85%] Linking CXX shared library libcust_tf_parsers.so [ 85%] Built target cust_tf_parsers [ 92%] Linking CXX shared library libcust_opsproto_rt2.0.so [100%] Linking CXX shared library libcust_opmaster_rt2.0.so [100%] Built target cust_op_proto [100%] Built target cust_optiling [100%] Built target optiling_compat Run CPack packaging tool... CPack: Create package using External CPack: Install projects CPack: - Run preinstall target for: opp CPack: - Install project: opp [] CPack: Create package About to compress 596 KB of data... Adding files to archive named "custom_opp_euleros_aarch64.run"... ./help.info ./install.sh ./packages/vendors/customize/framework/tensorflow/libcust_tf_parsers.so ./packages/vendors/customize/framework/tensorflow/npu_supported_ops.json ./packages/vendors/customize/op_api/include/aclnn_sigmoid_custom.h ./packages/vendors/customize/op_api/lib/libcust_opapi.so ./packages/vendors/customize/op_impl/ai_core/tbe/config/ascend310b/aic-ascend310b-ops-info.json ./packages/vendors/customize/op_impl/ai_core/tbe/config/ascend910b/aic-ascend910b-ops-info.json ./packages/vendors/customize/op_impl/ai_core/tbe/customize_impl/dynamic/sigmoid_custom.cpp ./packages/vendors/customize/op_impl/ai_core/tbe/customize_impl/dynamic/sigmoid_custom.py ./packages/vendors/customize/op_impl/ai_core/tbe/kernel/ascend310b/sigmoid_custom/ ./packages/vendors/customize/op_impl/ai_core/tbe/kernel/ascend910b/sigmoid_custom/ ./packages/vendors/customize/op_impl/ai_core/tbe/kernel/config/ascend310b/ ./packages/vendors/customize/op_impl/ai_core/tbe/kernel/config/ascend910b/ ./packages/vendors/customize/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/libcust_opmaster_rt2.0.so ./packages/vendors/customize/op_impl/ai_core/tbe/op_tiling/liboptiling.so ./packages/vendors/customize/op_proto/inc/op_proto.h ./packages/vendors/customize/op_proto/lib/linux/aarch64/libcust_opsproto_rt2.0.so ./packages/vendors/customize/version.info ./upgrade.sh CRC: 1499435367 SHA256: 132259459f6127571b863c6b1d537ab0b4bc03d419dcefd6159d37927d379782 Skipping md5sum at user request Self-extractable archive "custom_opp_euleros_aarch64.run" successfully created. Copy /home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out/_CPack_Packages/Linux/External/custom_opp_euleros_aarch64.run/custom_opp_euleros_aarch64.run to /home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out/ CPack: - package: /home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out/custom_opp_euleros_aarch64.run.json generated. CPack: - package: /home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out/custom_opp_euleros_aarch64.run generated. Verifying archive integrity... 100% SHA256 checksums are OK. All good. Uncompressing version:1.0 100% [ops_custom] [2025-06-23 23:33:13] [INFO] copy uninstall sh success [ops_custom] [2025-06-23 23:33:13] [INFO] upgrade framework tensorflow [ops_custom] [2025-06-23 23:33:13] [INFO] replace or merge old ops framework files .g..... [ops_custom] [2025-06-23 23:33:13] copy new ops framework files ...... [ops_custom] [2025-06-23 23:33:14] [INFO] upgrade op proto inc lib [ops_custom] [2025-06-23 23:33:14] [INFO] replace or merge old ops op_proto files .g..... [ops_custom] [2025-06-23 23:33:14] copy new ops op_proto files ...... [ops_custom] [2025-06-23 23:33:14] [INFO] upgrade op impl ai_core [ops_custom] [2025-06-23 23:33:14] [INFO] replace or merge old ops op_impl files .g..... [ops_custom] [2025-06-23 23:33:14] copy new ops op_impl files ...... [ops_custom] [2025-06-23 23:33:14] [INFO] upgrade op api include lib [ops_custom] [2025-06-23 23:33:14] [INFO] replace or merge old ops op_api files .g..... [ops_custom] [2025-06-23 23:33:14] copy new ops op_api files ...... [ops_custom] [2025-06-23 23:33:14] [INFO] upgrade version.info [ops_custom] [2025-06-23 23:33:14] copy new version.info files ...... [ops_custom] [2025-06-23 23:33:14] [INFO] no need to upgrade custom.proto files [ops_custom] [2025-06-23 23:33:14] [INFO] using requirements: when custom module install finished or before you run the custom module, execute the command [ export LD_LIBRARY_PATH=/home/ma-user/Ascend/ascend-toolkit/latest/opp/vendors/customize/op_api/lib/:${LD_LIBRARY_PATH} ] to set the environment path SUCCESS [100%] Built target ascendc_impl_gen [100%] Built target ascendc_bin_ascend910b_sigmoid_custom_copy [100%] Built target ascendc_bin_ascend310b_sigmoid_custom_copy [100%] Built target ascendc_bin_ascend910b [100%] Built target ascendc_bin_ascend310b [Ascend310B1] Generating SigmoidCustom_a3c9eb1f1b227778957282b95ed93786 ... [Ascend910B1] Generating SigmoidCustom_a3c9eb1f1b227778957282b95ed93786 ... [ERROR] TBE(123105,python3):2025-06-23-23:33:18.348.085 [../../../opc_tool/opc_common.py:218][_log_error] Traceback (most recent call last): File "/home/ma-user/Ascend/ascend-toolkit/8.0.RC1.alpha002/python/site-packages/opc_tool/op_compilation.py", line 417, in get_op_func_attr opm = importlib.import_module(op_module) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 680, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 850, in exec_module [ERROR] TBE(123105,python3):2025-06-23-23:33:18.348.235 [../../../opc_tool/opc_common.py:218][_log_error] File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out/op_kernel/binary/ascend910b/src/SigmoidCustom.py", line 14, in <module> from tbe.tikcpp.compile_op import CommonUtility, AscendCLogLevel ImportError: cannot import name 'CommonUtility' from 'tbe.tikcpp.compile_op' (/home/ma-user/Ascend/ascend-toolkit/8.0.RC1.alpha002/python/site-packages/tbe/tikcpp/compile_op.py) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/ma-user/Ascend/ascend-toolkit/8.0.RC1.alpha002/python/site-packages/opc_tool/op_compilation.py", line 559, in single_op_compilation self.__op_compile_by_json_info(json_dict) [ERROR] TBE(123105,python3):2025-06-23-23:33:18.348.409 [../../../opc_tool/op_compilation.py:562][single_op_compilation] Compile failed, reason is: import src.SigmoidCustom error, reason:cannot import name 'CommonUtility' from 'tbe.tikcpp.compile_op' (/home/ma-user/Ascend/ascend-toolkit/8.0.RC1.alpha002/python/site-packages/tbe/tikcpp/compile_op.py).. [ERROR] TBE(123105,python3):2025-06-23-23:33:18.348.498 [../../../opc_tool/opc.py:696][main] Opc tool compile failed. Opc tool start working now, please wait for a moment. /home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out/op_kernel/binary/ascend910b/bin/sigmoid_custom/SigmoidCustom_a3c9eb1f1b227778957282b95ed93786.json not generated! gmake[3]: *** [op_kernel/CMakeFiles/ascendc_bin_ascend910b_sigmoid_custom_0.dir/build.make:70: op_kernel/CMakeFiles/ascendc_bin_ascend910b_sigmoid_custom_0] Error 1 gmake[2]: *** [CMakeFiles/Makefile2:645: op_kernel/CMakeFiles/ascendc_bin_ascend910b_sigmoid_custom_0.dir/all] Error 2 gmake[2]: *** Waiting for unfinished jobs.... [ERROR] TBE(123103,python3):2025-06-23-23:33:18.302.945 [../../../opc_tool/opc_common.py:218][_log_error] Traceback (most recent call last): File "/home/ma-user/Ascend/ascend-toolkit/8.0.RC1.alpha002/python/site-packages/opc_tool/op_compilation.py", line 417, in get_op_func_attr opm = importlib.import_module(op_module) File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 680, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 850, in exec_module [ERROR] TBE(123103,python3):2025-06-23-23:33:18.303.138 [../../../opc_tool/opc_common.py:218][_log_error] File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out/op_kernel/binary/ascend310b/src/SigmoidCustom.py", line 14, in <module> from tbe.tikcpp.compile_op import CommonUtility, AscendCLogLevel ImportError: cannot import name 'CommonUtility' from 'tbe.tikcpp.compile_op' (/home/ma-user/Ascend/ascend-toolkit/8.0.RC1.alpha002/python/site-packages/tbe/tikcpp/compile_op.py) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/ma-user/Ascend/ascend-toolkit/8.0.RC1.alpha002/python/site-packages/opc_tool/op_compilation.py", line 559, in single_op_compilation self.__op_compile_by_json_info(json_dict) [ERROR] TBE(123103,python3):2025-06-23-23:33:18.303.315 [../../../opc_tool/op_compilation.py:562][single_op_compilation] Compile failed, reason is: import src.SigmoidCustom error, reason:cannot import name 'CommonUtility' from 'tbe.tikcpp.compile_op' (/home/ma-user/Ascend/ascend-toolkit/8.0.RC1.alpha002/python/site-packages/tbe/tikcpp/compile_op.py).. [ERROR] TBE(123103,python3):2025-06-23-23:33:18.303.558 [../../../opc_tool/opc.py:696][main] Opc tool compile failed. Opc tool start working now, please wait for a moment. /home/ma-user/work/SigmoidCustom/SigmoidCustom/build_out/op_kernel/binary/ascend310b/bin/sigmoid_custom/SigmoidCustom_a3c9eb1f1b227778957282b95ed93786.json not generated! gmake[3]: *** [op_kernel/CMakeFiles/ascendc_bin_ascend310b_sigmoid_custom_0.dir/build.make:70: op_kernel/CMakeFiles/ascendc_bin_ascend310b_sigmoid_custom_0] Error 1 gmake[2]: *** [CMakeFiles/Makefile2:514: op_kernel/CMakeFiles/ascendc_bin_ascend310b_sigmoid_custom_0.dir/all] Error 2 gmake[1]: *** [CMakeFiles/Makefile2:416: op_kernel/CMakeFiles/binary.dir/rule] Error 2 gmake: *** [Makefile:306: binary] Error 2 [ERROR] Kernel compile failed, the run package will not be generated.
最新发布
06-24
<think>我们正在处理一个关于华为云AscendC算子开发的问题,具体是在构建SigmoidCustom算子时遇到TBE编译错误:ImportError:cannotimportname'CommonUtility'from'tbe.tikcpp.compile_op'根据引用[1]中的错误信息,我们看到类似的问题:在运行TBE算子测试时出现了ModuleNotFoundError:Nomodulenamed'absl'。这表明环境配置可能有问题,特别是Python依赖项。引用[2]提到:MindStudio版本和CANN版本必须一致,否则会导致算子工程创建出错。因此,我们需要检查版本兼容性。引用[3]描述了TBE算子的目录结构,其中impl目录存放算子实现代码,testcase目录存放测试用例。这有助于我们定位代码位置。引用[4]介绍了TBE算子的背景,它是基于TVM扩展的,使用PythonAPI开发。因此,我们可以从以下步骤进行排查:1.检查MindStudio和CANN版本是否匹配。根据引用[2],我们需要确认安装的MindStudio版本和CANN版本是否一致。如果不一致,需要重新安装匹配的版本。2.检查Python依赖项。根据引用[1]中出现的absl模块缺失错误,我们需要确保所有依赖的Python模块都已安装。同样,当前错误是导入CommonUtility失败,这可能也是由于依赖问题或环境配置错误。3.检查TBE算子代码实现和编译环境。确保算子代码(位于impl目录)正确,并且编译环境配置无误。4.尝试重新编译或重新创建工程。如果环境配置正确,但问题仍然存在,可以尝试重新创建算子工程或重新编译。5.检查AscendToolkit的安装路径。确保环境变量设置正确,特别是PYTHONPATH,使其包含AscendToolkit的site-packages目录。具体步骤:步骤1:检查版本兼容性在MindStudio中查看其版本,并在终端中运行命令查看CANN版本,例如:/usr/local/Ascend/ascend-toolkit/latest/atc/bin/atc--version确保两者版本一致(如MindStudio5.0.RC3对应CANN6.0.RC1)。步骤2:安装缺失的Python模块根据错误信息,可能需要安装absl-py模块,使用pip安装:pipinstallabsl-py同时,检查tbe.tikcpp.compile_op模块是否存在。可以尝试在Python环境中导入:python-c"fromtbe.tikcpp.compile_opimportCommonUtility"如果失败,则可能是AscendToolkit安装不完整或路径未设置。步骤3:设置环境变量确保环境变量PYTHONPATH包含AscendToolkit的site-packages路径,例如:exportPYTHONPATH=/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/built-in/ai_core/tbe:$PYTHONPATH注意:具体路径可能因安装位置和版本而异。步骤4:重新编译在算子工程目录下,尝试清理并重新构建:./build.shclean./build.sh步骤5:检查算子实现文件根据引用[3],确保算子实现文件(如impl/sigmoid_custom.py)中的代码正确,特别是没有语法错误或导入错误。如果以上步骤无法解决,可能是AscendToolkit安装损坏,建议重新安装匹配版本的CANN。相关问题:1.如何检查MindStudio和CANN版本是否匹配?2.在AscendC算子开发中,如何正确配置Python环境变量?3.如何安装TBE算子开发所需的Python依赖项?4.在构建TBE算子时遇到导入错误,有哪些常见的解决方法?5.如何重新安装AscendToolkit(CANN)?由于用户的问题描述中提到了具体的错误信息,我们重点解决CommonUtility导入失败的问题。根据经验,该问题通常由环境配置错误或版本不匹配导致。注意:CommonUtility是TBE编译工具中的一个模块,它的缺失可能是由于TBE相关代码未正确安装或环境变量未配置导致。因此,我们建议用户按照上述步骤进行排查,特别是步骤1和步骤2。</think>### Ascend C算子开发:解决TBE编译错误ImportError #### 问题分析 在构建SigmoidCustom算子时出现`ImportError: cannot import name 'CommonUtility' from 'tbe.tikcpp.compile_op'`错误,通常由以下原因导致: 1. **环境版本不匹配** MindStudio与CANN版本不一致会导致核心模块缺失[^2]: ```bash # 检查版本一致性 mindstudio --version # MindStudio版本 /usr/local/Ascend/ascend-toolkit/latest/fwkacllib/python/site-packages/te/version.py # CANN版本 ``` 2. **Python依赖项缺失** 类似引用[1]中`absl`模块缺失的错误,表明Python环境不完整。 3. **环境变量配置错误** TBE核心路径未加入`PYTHONPATH`,导致解释器找不到`CommonUtility`模块。 #### 解决方案 ##### 步骤1:验证环境兼容性 1. 确认MindStudio与CANN版本匹配(参考官方兼容矩阵): - MindStudio 5.0.RC3 → CANN 6.0.RC1 - 若不匹配,重新下载对应版本[^2] 2. 检查Ascend Toolkit安装完整性: ```bash ls /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/built-in/ai_core/tbe/tikcpp # 确认存在 compile_op.py 文件 ``` ##### 步骤2:修复Python环境 1. 安装关键依赖: ```bash pip install absl-py pyyaml numpy decorator ``` 2. 配置环境变量(添加到`~/.bashrc`): ```bash export PYTHONPATH=/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/built-in/ai_core/tbe:$PYTHONPATH export LD_LIBRARY_PATH=/usr/local/Ascend/ascend-toolkit/latest/fwkacllib/lib64:$LD_LIBRARY_PATH source ~/.bashrc ``` ##### 步骤3:重建算子工程 1. 清理构建缓存: ```bash cd /path/to/SigmoidCustom ./build.sh clean rm -rf out/ # 删除错误生成的中间文件 ``` 2. 重新编译: ```bash ./build.sh ``` ##### 步骤4:验证TBE环境 在Python环境中测试导入: ```python python -c "from tbe.tikcpp.compile_op import CommonUtility; print('Import success!')" ``` 若仍失败,需重新安装Ascend Toolkit。 #### 预防措施 1. **工程结构标准化**(参考引用[3]): ``` tbe/ ├── impl/ │ └── sigmoid_custom.py # 算子实现 ├── testcase/ │ └── ms2.0_test/ # MindSpore测试用例 └── op_info_cfg/ # 算子信息配置 ``` 2. 使用官方模板创建算子: ```bash mindstudio -> File -> New -> Ascend Operator Development ``` #### 根本原因说明 `CommonUtility`是TBE(Tensor Boost Engine)的核心编译工具类[^4],负责将Python代码转换为NPU可执行指令。当Ascend Toolkit安装不完整或环境路径配置错误时,该模块无法被正确加载,导致构建中断。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值