OpenDataCube安装教程（二）——DataCube安装配置

置顶气运2020

已于 2022-02-24 11:29:15 修改

阅读量1.5k

点赞数 1

分类专栏： Open Data Cube PostgreSQL 文章标签： ubuntu linux centos

于 2021-08-30 14:54:05 首次发布

本文链接：https://blog.youkuaiyun.com/weixin_46307478/article/details/119997018

版权

PostgreSQL 同时被 2 个专栏收录

5 篇文章

订阅专栏

Open Data Cube

3 篇文章

订阅专栏

二、OpenDataCube的安装

教程：

Ubuntu — Open Data Cube 1.8-FIXME documentation (datacube-core.readthedocs.io)

依赖环境：

PostgreSQL 9.5+
Python 3.6+

1.依赖环境安装

（不同系统，依赖名称可能不同）

提示：官网给的是针对Ubuntu的，自己是在Centos上，名称不太一样，故不确定安装的依赖是否正确，建议能安装的都安装上。

yum install -y \
  autoconf automake build-essential make cmake \
  graphviz \
  python3-venv \
  python3-dev \
  libpq-dev \
  libyaml-dev \
  libnetcdf-dev \
  libudunits2-dev
 
 
(datacube) [root@song agdc-v2]# yum install -y gcc
(datacube) [root@song agdc-v2]# yum install gcc-c++

#提示：Centos与Ubuntu安装的必要依赖的名称不同
#以下是在Centos下的安装
(datacube) [root@song agdc-v2]# yum install python-devel
(datacube) [root@song agdc-v2]# yum install libevent-devel

2.下载源码

git clone https://github.com/opendatacube/datacube-core.git

git命令的安装：sudo yum install -y git

3.创建虚拟Python环境

以下两种方式二选一即可，推荐minconda

方式一：使用conda

(base) [root@song ~]# conda config --add channels conda-forge		#添加conda-forge镜像源
(base) [root@song ~]# conda create --name odc python=3.6 datacube	#创建python虚拟环境，过程中安装datacube包

方式二：使用Pyenv

pyenv="${HOME}/.envs/odc"  # Change to suit your needs
mkdir -p "${pyenv}"
python3 -m venv "${pyenv}"
source "${pyenv}/bin/activate"
pip install -U pip wheel cython numpy
pip install -e '.[dev]'
pip install flake8 mypy pylint autoflake black

4.激活python环境，安装必要的包

(base) [root@song ~]# conda activate odc
(odc) [root@song ~]# conda install jupyter matplotlib scipy

(odc) [root@song software]# cd datacube-core
(odc) [root@song datacube-core]# pip install --upgrade -e .

提示：

1）一定是在激活的odc环境下进行安装测试！！

2）若安装过程中遇到某个库无法安装的问题，考虑使用conda install 单独安装，如下：

解决方法：
(base) [root@song datacube-core]# conda install psycopg2

5.安装Datacube

(odc) [root@song ~]# cd software/datacube-core/
(odc) [root@song datacube-core]# python setup.py install

6.创建用户与数据库

(base) [root@song ~]# sudo -i -u postgres
-bash-4.2$ createuser --superuser odctest
-bash-4.2$ psql -c "ALTER USER odctest WITH PASSWORD '123456';"		#创建新用户
ALTER ROLE
-bash-4.2$ psql -c "CREATE DATABASE datacube OWNER odctest;"		#创建用户数据库
CREATE DATABASE

(base) [root@song ~]# sudo adduser odctest						#Linux下创建与数据库用户 同名 的用户
(base) [root@song ~]# sudo passwd odctest						#设置用户密码
Changing password for user odctest.
New password: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.
(base) [root@song ~]# su - odctest								#切换到数据库用户
[odctest@song ~]$ psql -d datacube								#以odctest的身份连接数据库datacube
psql (12.8)
Type "help" for help.

datacube=#

7.创建配置文件

(odc) [root@song datacube-core]# vim  ~/.datacube.conf   #原来不存在该配置文件，则直接创建，复制下面内容

[datacube]
db_database: datacube

# A blank host will use a local socket. Specify a hostname (such as localhost) to use TCP.
db_hostname:localhost

# Credentials are optional: you might have other Postgres authentication configured.
# The default username otherwise is the current user id.
db_username:odctest
db_password:123456


(odc) [root@song ~]# cd ~/
(odc) [root@song ~]# chmod 777 .datacube.conf 			#赋予执行权限

8.初始化数据库

(odc) [root@song ~]# cd software/datacube-core/
(odc) [root@song datacube-core]# datacube -v system init
/root/envs/miniconda3/envs/odc/lib/python3.6/site-packages/distributed/config.py:63: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  config.update(yaml.load(text))
2021-08-30 14:29:48,548 4550 datacube INFO Running datacube command: /root/envs/miniconda3/envs/odc/bin/datacube -v system init
Initialising database...
2021-08-30 14:29:48,609 4550 datacube.drivers.postgres._core INFO Ensuring user roles.
2021-08-30 14:29:48,626 4550 datacube.drivers.postgres._core INFO Creating schema.
2021-08-30 14:29:48,628 4550 datacube.drivers.postgres._core INFO Creating tables.
2021-08-30 14:29:48,695 4550 datacube.drivers.postgres._core INFO Creating triggers.
2021-08-30 14:29:48,725 4550 datacube.drivers.postgres._core INFO Creating added column.
2021-08-30 14:29:48,750 4550 datacube.drivers.postgres._core INFO Adding role grants.
2021-08-30 14:29:48,767 4550 datacube.index.index INFO Adding default metadata types.
Created.
Checking indexes/views.
2021-08-30 14:29:49,135 4550 datacube.drivers.postgres._api INFO Checking dynamic views/indexes. (rebuild views=True, indexes=False)
Done.

9.进行测试

单元测试

(odc) [root@song datacube-core]# pip install --upgrade -e '.[test]'
...
Installing collected packages: datacube
  Attempting uninstall: datacube
    Found existing installation: datacube 0.0.0
    Uninstalling datacube-0.0.0:
      Successfully uninstalled datacube-0.0.0
  Running setup.py develop for datacube
Successfully installed datacube-1.8.6.dev0+g3a49f78.d20210830

集成测试？

./check-code.sh integration_tests

!过程中遇到的问题：

1.运行setup.py时失败，由于使用miniconda创建的虚拟环境，而setup.py脚本中指定的python路径是默认的，需要删除或者修改。

2.运行setup.py时Setup script exited with error: command ‘gcc’ failed with exit status 1

(datacube) [root@song agdc-v2]# yum install -y gcc
(datacube) [root@song agdc-v2]# yum install gcc-c++

关于command ‘gcc’ failed with exit status 1 解决方法 - gerrydeng - 博客园 (cnblogs.com)

3.进行数据库初始化时，缺少配置文件Error: No datacube config found

解决方法：创建配置文件

参考官方教程：

Database Setup — Open Data Cube 1.4.1+0.gadcda1a documentation (datacube-core.readthedocs.io)

(odc) [root@song datacube-core]# vim  ~/.datacube.conf   #原来不存在该配置文件，则直接创建，复制下面内容

[datacube]
db_database: datacube

# A blank host will use a local socket. Specify a hostname (such as localhost) to use TCP.
db_hostname:localhost

# Credentials are optional: you might have other Postgres authentication configured.
# The default username otherwise is the current user id.
db_username:odctest
db_password:123456


(odc) [root@song ~]# cd ~/
(odc) [root@song ~]# chmod 777 .datacube.conf 			#赋予执行权限

4.数据库初始化时，认证失败 sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: Ident authentication failed for user “postgres”

解决方式：

(odc) [root@song ~]# cp /var/lib/pgsql/12/data/pg_hba.conf /var/lib/pgsql/12/data/pg_hba.conf.bak		#先做备份
(odc) [root@song ~]# vim /var/lib/pgsql/12/data/pg_hba.conf			#把method全部改为trust


# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
# IPv6 local connections:
host    all             all             ::1/128                 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     trust
host    replication     all             127.0.0.1/32            trust
host    replication     all             ::1/128                 trust
host    all             all             0.0.0.0/0               trust


(odc) [root@song ~]# systemctl restart postgresql-12		#重启服务