deeprec编译

DeepRec编译1

容器内下载编译

docker pull alideeprec/deeprec-build:deeprec-dev-cpu-py36-ubuntu18.04
docker run --net host -it alideeprec/deeprec-build:deeprec-dev-cpu-py36-ubuntu18.04
#进入容器
git clone https://github.com/DeepRec-AI/DeepRec.git
cd DeepRec/
./configure
git checkout deeprec2304
bazel build -c opt --config=opt //tensorflow/tools/pip_package:build_pip_package
wget https://github.com/nelhage/rules_boost/archive/4ee400beca08f524e7ea3be3ca41cce34454272f.tar.gz
cp 4ee400beca08f524e7ea3be3ca41cce34454272f.tar.gz /root/.cache/bazel/_bazel_root/35e4a563d8d022421e7c53a7646fe840/external/com_github_nelhage_rules_boost/
wget https://storage.googleapis.com/dataset-uploader/criteo-kaggle/large_version/train.csv
wget https://storage.googleapis.com/dataset-uploader/criteo-kaggle/large_version/eval.csv

编译2

下载代码后，容器内编译

docker run --net=host -it -v /data1/deeprec:/root/deeprec alideeprec/deeprec-build:deeprec-dev-cpu-py36-ubuntu18.04
cd /root/deeprec 
# chmod -R 777 configure
# chmod -R 777 tensorflow/tools/pip_package/build_pip_package.sh
./configure
WARNING: current bazel installation is not a release version.
Make sure you are running at least bazel 0.26.1
Do you wish to build TensorFlow with XLA JIT support? [Y/n]: n
No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with STAR support? [Y/n]: y
STAR support will be enabled for TensorFlow.

Do you wish to build TensorFlow with PMEM support? [y/N]: n
No PMEM support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Parquet Dataset support? [Y/n]: y
Parquet Dataset support will be enabled for TensorFlow.

Do you wish to build TensorFlow with API COMPATIBLE WITH TF 1150 support? [y/N]: n
No API COMPATIBLE WITH TF 1150 support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with MPI support? [y/N]: n
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:


Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
        --config=mkl_threadpool # Build with oneDNN support.
        --config=mkl_aarch64    # Build with oneDNN and Compute Library for the Arm Architecture (ACL).
        --config=monolithic     # Config for mostly static monolithic build.
        --config=gdr            # Build with GDR support.
        --config=verbs          # Build with libverbs support.
        --config=ngraph         # Build with Intel nGraph support.
        --config=numa           # Build with NUMA support.
        --config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
        --config=v2             # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
        --config=noaws          # Disable AWS S3 filesystem support.
        --config=nogcp          # Disable GCP support.
        --config=nohdfs         # Disable HDFS support.
        --config=noignite       # Disable Apache Ignite support.
        --config=nokafka        # Disable Apache Kafka support.
        --config=nonccl         # Disable NVIDIA NCCL support.
Configuration finished
bazel build -c opt --config=opt //tensorflow/tools/pip_package:build_pip_package

# Fetching ...azel_rules_docker; Cloning 251f6a68b439744094faff800cd029798edf9faa of https://github.com/bazelbuild/rules_docker.git 96s 偶发性，貌似跟网络有关
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
pip3 install tensorflow-1.15.5+deeprec2304-cp36-cp36m-linux_x86_64.whl

编译jemalloc

下载解压安装包

wget https://github.com/jemalloc/jemalloc/releases/download/5.1.0/jemalloc-5.1.0.tar.bz2
tar xjvf jemalloc-5.1.0.tar.bz2

编译

cd jemalloc-5.1.0/
./configure
make
ls ./lib
libjemalloc.a  libjemalloc_pic.a  libjemalloc.so  libjemalloc.so.2

dockerfile打包镜像

dcokerfile参数

-f 指定dockerfile文件 
-t 打标签
. 构建镜像时的上下文环境

# 路径下不要放置其他文件
docker build -t deeprec:v3 -f Dockerfile.tf4-devel-py3.6-ubuntu18.04 .
#上传打包好的镜像到镜像库
#给镜像打上公司镜像仓库标签
docker image tag alideeprec/deeprec-release:deeprec2304-cpu-py38-ubuntu20.04 company/search_rec/deeprec-dev-cpu-py38-ubuntu20.04
#登陆公司镜像仓库
docker login company.com
#push到公司镜像仓库
docker push company/deeprec-dev-cpu-py38-ubuntu20.04

tfserving编译

docker pull alideeprec/deeprec-build:deeprec-dev-cpu-py38-ubuntu20.04
git clone https://github.com/DeepRec-AI/serving.git
cd serving/
git checkout deeprec2304
docker run --net host -v /data1/minisearch/serving:/serving alideeprec/deeprec-build:deeprec-dev-cpu-py38-ubuntu20.04
cd /serving
bazel build -c opt tensorflow_serving/...
# 漫长等待，大概一个半小时，可能有如下错误，则需要安装automake
./autogen.sh: aclocal: not found
apt-get install automake

镜像启动

docker run -t -p 8501:8501 -p 8500:8500 -v /models/deepfm:/models/deepfm/ -e MODEL_NAME=deepfm tensorflow/serving:1.15.0

./autogen.sh: aclocal: not found
dockerfile文章1
dockerfile文章2