Bazel 创建repository_rule

本文介绍了如何利用Bazel的repository_rule功能自定义规则,解决在无法修改远程库源码的情况下,根据需求动态选择静态库或动态库的问题。通过示例展示了如何创建一个名为module_a_repo的规则,使得WORKSPACE文件能够根据传入参数动态改变库的依赖类型。

Bazel的repository_rule

一直使用Bazel内建的repository (git_repository, http_archive, etc),但实际上bazel提供了自定义的repository_rule给我们自定义repository的rule,以提高我们定义库的方式。

一个例子

假如在一个远端git上有一个c++的库A,它是一个预编译过的库。也就是说它包含一个lib目录,里面有.a和.so文件,但是在这个库A的BUILD文件里,它是这么写的:

cc_library(
    name = "module_a",
    hdrs = glob(["include/*.h"]),
    srcs = glob(["lib/*.a"]),
)

很显然,这个库只提供了静态库,并不提供动态库。

突然有一天由于某些特殊原因,我们需要使用这个库里的so,而不是.a,我们会怎么做?

最直接的思路就是在库A里增加一个cc_library的rule “libmodule_a”,这个rule的srcs里只包含.so。然后把我们自己库里的所有依赖都改个遍,把依赖@A//:module_a 都换成@A//:libmodule_a。

但是非常不幸,对于库A我们没有修改权限,并且对于本地编译来说,我们既要提供依赖.a的编译方式,也需要提供.so的编译方式。

怎么办?可能另一个思路是,对于库A我们使用new_git_repository的rule,然后对其指定我们本地的有动态库rule的BUILD文件。然后我们通过config_setting指定编译参数,在本地进行依赖时通过config_setting选择相应的module_a的rule进行依赖。

这是一种思路,但是如果我们自己的库B的依赖库C,D,E也同时依赖了@A//:module_a,我们可以有权限更改自己的库B进行config_setting select deps rule,但是我们如何保证其他库的依赖也更改呢?

另一种思路:使用repository_rule对库A进行改造,让它根据我们传入的参数对库A的BUILD文件进行改变。例如我们在workspace.bzl里自定义一个module_a_repo的rule:

def _git_clone_or_update(ctx):
    if ((ctx.attr.tag == "" and ctx.attr.commit == "") or
        (ctx.attr.tag != "" and ctx.attr.commit != "")):
        ctx.fail("Exactly one of commit or tag should be provided")
    if ctx.attr.commit != "":
        ref = ctx.attr.commit
    else:
        ref = "tags/" + ctx.attr.tag
    st = ctx.execute(["bash", '-c', """
set -ex
( cd {working_dir} &&
    if ! (cd '{dir}' && git rev-parse --git-dir) >/dev/null 2>&1; then
        rm -rf '{dir}'
        git clone '{remote}' '{dir}'
    fi
    cd '{dir}'
    git reset --hard {ref} || (git fetch && git reset --hard {ref})
    git clean -xdf)
    """.format(
        working_dir=ctx.path(".").dirname,
        dir=ctx.path("."),
        remote=ctx.attr.remote,
        ref=ref,
    )])
    if st.return_code != 0:
        fail("error cloning %s:\n%s" % (ctx.name, st.stderr))

def _module_a_repo_implement(ctx):
    _git_clone_or_update(ctx)
    BUILD_CONTENT="""
cc_library(
    name = "module_a",
    hdrs = ["include/*.h"],
    srcs = ["lib/*.a"],
    includes = ["include"],
    visibility = [
        "//visibility:public",
    ],
)
    """
    if ctx.attr.mode == "dynamic":
        BUILD_CONTENT = BUILD_CONTENT.replace("lib/*.a", 'lib/*.so')
    ctx.file('BUILD', content=BUILD_CONTENT)


module_a_repo = repository_rule(
    implementation = _module_a_repo_implement,
    attrs = {
        "remote": attr.string(mandatory=True),
        "commit": attr.string(default=""),
        "tag": attr.string(default=""),
        "mode": attr.string(default="static")
    }
)

那个在WORKSPACE文件里可以直接这样写:

load("//:workspace.bzl", "module_a_repo")
module_a_repo(
    name="A",
    remote="git@git.xxx.com:A/A.git",
    commit="some_commit",
    mode="static"
)

如果我们想要编译时依赖A的动态库,那么可以把mode="static"改成mode="dynamic", 这样的操作在shell脚本里就非常容易实现了。

FAILED: out_krn/target/product/mgk_64_k61_auto_vm_uos/obj/KLEAF_OBJ/dist/kernel_device_modules-6.1/mgk_64_k61_kernel_aarch64.userdebug/Image.lz4 out_krn/target/product/mgk_64_k61_auto_vm_uos/obj/KLEAF_OB J/dist/kernel_device_modules-6.1/mgk_64_k61_kernel_aarch64.userdebug/modules.builtin /bin/bash -c "(cd kernel && export BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 DEFCONFIG_OVERLAYS=\"auto.config wifionly.config nebula_vm.config uos.config\" KERNEL_VERSION=kernel-6.1 && tools/bazel --output_roo t=/mnt/sda1/WT-8678/MTK_MP/android/out_krn/target/product/mgk_64_k61_auto_vm_uos/obj/KLEAF_OBJ --output_base=/mnt/sda1/WT-8678/MTK_MP/android/out_krn/target/product/mgk_64_k61_auto_vm_uos/obj/KLEAF_OBJ/b azel/output_user_root/output_base build --//build/bazel_mgk_rules:kernel_version=6.1 --experimental_writable_outputs //kernel_device_modules-6.1:mgk_64_k61_customer_modules_install.userdebug ) && (cd ker nel && export BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 DEFCONFIG_OVERLAYS=\"auto.config wifionly.config nebula_vm.config uos.config\" KERNEL_VERSION=kernel-6.1 && tools/bazel --output_root=/mnt/sda1/WT-8678/M TK_MP/android/out_krn/target/product/mgk_64_k61_auto_vm_uos/obj/KLEAF_OBJ --output_base=/mnt/sda1/WT-8678/MTK_MP/android/out_krn/target/product/mgk_64_k61_auto_vm_uos/obj/KLEAF_OBJ/bazel/output_user_root /output_base run --//build/bazel_mgk_rules:kernel_version=6.1 --experimental_writable_outputs --nokmi_symbol_list_violations_check //kernel_device_modules-6.1:mgk_64_k61_customer_dist.userdebug -- --dist _dir=/mnt/sda1/WT-8678/MTK_MP/android/out_krn/target/product/mgk_64_k61_auto_vm_uos/obj/KLEAF_OBJ/dist )" Starting local Bazel server and connecting to it... INFO: Repository kernel_toolchain_info instantiated at: /mnt/sda1/WT-8678/MTK_MP/android/kernel/WORKSPACE:17:23: in <toplevel> /mnt/sda1/WT-8678/MTK_MP/android/kernel/build/kernel/kleaf/workspace.bzl:79:19: in define_kleaf_workspace Repository rule key_value_repo defined at: /mnt/sda1/WT-8678/MTK_MP/android/kernel/build/kernel/kleaf/key_value_repo.bzl:32:33: in <toplevel> ERROR: An error occurred during the fetch of repository 'kernel_toolchain_info': Traceback (most recent call last): File "/mnt/sda1/WT-8678/MTK_MP/android/kernel/build/kernel/kleaf/key_value_repo.bzl", line 8, column 42, in _impl raw_content = repository_ctx.read(src) Error in read: Unable to load package for //common:build.config.constants: BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package. - /mnt/sda1/WT-8678/MTK_MP/android/kernel/common ERROR: /mnt/sda1/WT-8678/MTK_MP/android/kernel/WORKSPACE:17:23: fetching key_value_repo rule //external:kernel_toolchain_info: Traceback (most recent call last): File "/mnt/sda1/WT-8678/MTK_MP/android/kernel/build/kernel/kleaf/key_value_repo.bzl", line 8, column 42, in _impl raw_content = repository_ctx.read(src) Error in read: Unable to load package for //common:build.config.constants: BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package. - /mnt/sda1/WT-8678/MTK_MP/android/kernel/common ERROR: Skipping '//build/bazel_mgk_rules:kernel_version': no such package '@kernel_toolchain_info//': Unable to load package for //common:build.config.constants: BUILD file not found in any of the follow ing directories. Add a BUILD file to a directory to mark it as a package. - /mnt/sda1/WT-8678/MTK_MP/android/kernel/common WARNING: Target pattern parsing failed. ERROR: //build/bazel_mgk_rules:kernel_version :: Error loading option //build/bazel_mgk_rules:kernel_version: no such package '@kernel_toolchain_info//': Unable to load package for //common:build.config. constants: BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package. - /mnt/sda1/WT-8678/MTK_MP/android/kernel/common 15:09:20 ninja failed with: exit status 1
08-15
(PCC1) C:\Users\fuwei\Desktop\C409\model\Point-Cloud-Compression-main\compression>bazel --host_jvm_args="-Dhttps.proxy=http://127.0.0.1:7897" build --action_env PYTHON_BIN_PATH="C:\Users\fuwei\anaconda3\envs\PCC1\python.exe" //tensorflow_compression/pip_pkg:build_pip_pkg WARNING: Running Bazel server needs to be killed, because the startup options are different. Starting local Bazel server and connecting to it... INFO: Repository python_version_repo instantiated at: C:/users/fuwei/desktop/c409/model/point-cloud-compression-main/compression/WORKSPACE:26:25: in <toplevel> C:/users/fuwei/_bazel_fuwei/kswckth2/external/local_tsl/third_party/py/python_init_repositories.bzl:13:22: in python_init_repositories Repository rule python_repository defined at: C:/users/fuwei/_bazel_fuwei/kswckth2/external/local_tsl/third_party/py/python_repo.bzl:151:36: in <toplevel> ERROR: An error occurred during the fetch of repository 'python_version_repo': Traceback (most recent call last): File "C:/users/fuwei/_bazel_fuwei/kswckth2/external/local_tsl/third_party/py/python_repo.bzl", line 12, column 34, in _python_repository_impl version = _get_python_version(ctx) File "C:/users/fuwei/_bazel_fuwei/kswckth2/external/local_tsl/third_party/py/python_repo.bzl", line 87, column 21, in _get_python_version fail(""" Error in fail: Cannot match hermetic Python version to system Python version. System Python was not found. ERROR: C:/users/fuwei/desktop/c409/model/point-cloud-compression-main/compression/WORKSPACE:26:25: fetching python_repository rule //external:python_version_repo: Traceback (most recent call last): File "C:/users/fuwei/_bazel_fuwei/kswckth2/external/local_tsl/third_party/py/python_repo.bzl", line 12, column 34, in _python_repository_impl version = _get_python_version(ctx) File "C:/users/fuwei/_bazel_fuwei/kswckth2/external/local_tsl/third_party/py/python_repo.bzl", line 87, column 21, in _get_python_version fail(""" Error in fail: Cannot match hermetic Python version to system Python version. System Python was not found. ERROR: Error computing the main repository mapping: no such package '@python_version_repo//': Cannot match hermetic Python version to system Python version. System Python was not found. Loading:
10-30
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值