Docker TBase实战 Centos7+TBase-2.4
前言
Docker TBase实战
熟悉TBase安装过程及相关配置。
目标：简化TBase安装及配置过程
利用docker简化安装及配置过程主要用过开发环境或测试，生产或压测试环境不建议使用本文章搭建。官方文档
实现步骤
环境及相关条件说明
部署系统: centos-7.9.2009
Docker: docker 20.10.14, docker-compose v2.24.7
构建Master Node Docker镜像
构建准备
Docker 及 Docker-compose
TBase安装包
Centos-7.9.2009基础镜像
Docker 及 Docker-compose 请参看前言链接
TBase 下载
Centos-7.9.2009:docker pull registry.cn-hangzhou.aliyuncs.com/snake-group/centos-7.9.2009:1.0
构建镜像
启动基础镜像容器:tbase-base
docker run -d --name tbase-base --privileged=true registry.cn-hangzhou.aliyuncs.com/snake-group/centos-7.9.2009:1.0
可以用
docker ps查看启动情况
也可以用docker rm tbase-base删除容器重建
这里的参数--privileged=true不可少,不然无法安装软件
复制TBase 安装包到容器内
本例TBase安装包放在: D:\tbase\TBase-master.zip
docker cp D:\tbase\TBase-master.zip tbase-base:/opt
上面为把安装复制到容器内: /opt
容器内安装TBase
进入容器
docker exec -it tbase-base /bin/bash
安装依赖
yum install -y gcc gcc-c++ pcre-devel openssh-clients lrzsz bison.x86_64 bison-devel.x86_64 flex.x86_64 flex-devel.x86_64 readline readline-dev readline.x86_64 readline-devel.x86_64 zlib.x86_64 zlib-devel.x86_64 openssl-devel uuid uuid-devel git.x86_64 openssl openssh-server
添加tbase用户并指定用户目录及相关授权
mkdir -p /data/tbase && useradd -d /data/tbase tbase
chown -R tbase:tbase /data/tbase/
chown tbase:tbase /opt/TBase-master.zip && mv /opt/TBase-master.zip /data/tbase
passwd tbase #修改tbase密码为tbase,此外必有要设置密码
切换tbase用户
[root@2f9028b5f955 opt]# su tbase
bash-4.2$
此外显为
bash-4.2,交互不大友好，我们可以添加.bashrc文件到/data/tbase/下，内容如下# .bashrc # Source global if [ -f /etc/bashrc ]; then . /etc/bashrc fi export TBASE_HOME=/data/tbase/tbase_bin_v2.4 export PATH=$TBASE_HOME/bin:$PATH export LD_LIBRARY_PATH=$TBASE_HOME/lib:${LD_LIBRARY_PATH}此处还加了一此环境变量，后面做配置时使用
生效.bashrcsource /data/tbase/.bashrc
[root@2f9028b5f955 opt]# su tbase
bash-4.2$ vi /data/tbase/.bashrc
bash-4.2$ source /data/tbase/.bashrc
[tbase@2f9028b5f955 opt]$ ll
total 28304
-rwxr-xr-x 1 tbase tbase 28981049 12月 9 2022 TBase-master.zip
解压TBase安装包并编译安装
cd /data/tbase && unzip TBase-master.zip && cd TBase-master && ./configure --prefix=/data/tbase/tbase_bin_v2.4 --enable-user-switch --with-openssl --with-ossp-uuid CFLAGS=-g && make && make install
cd /data/tbase/TBase-master/contrib && make -sj && make install && rm -rf /opt/TBase-master.zip /data/tbase/TBase-master
安装检测
command -v pgxc_ctl
如果命令存在表式TBase 安装成功了
SSH互信配置：修改ssh_config
vi /etc/ssh/ssh_config
# 修改 59行 GSSAPIAuthentication yes => GSSAPIAuthentication no
# 保存退出
修改此要root权限，可以用
exit退出tbase用户
退出容器制作并镜像
docker commit -m 'tbase-base' tbase-base registry.cn-hangzhou.aliyuncs.com/snake-group/tbase-base:1.0
至此Master Node Docker镜像完成，可以用
docker images查看镜像
此镜像我已经上传到我的云仓库，同学们不想做镜像可以此镜像registry.cn-hangzhou.aliyuncs.com/snake-group/tbase-base:1.0
用docker-compose部署TBase
部署情况如下
| 节点名称 | 主机名称 | 路径 | 端口 |
|---|---|---|---|
| GTM Master | tbase01 | /data/tbase/pgxc/nodes/gtm | 20001 |
| GTM Slave | tbase02 | /data/tbase/pgxc/nodes/gtm | 20001 |
| CN cn001 | tbase01 | /data/tbase/pgxc/nodes/coord | 20004 |
| CN cn002 | tbase02 | /data/tbase/pgxc/nodes/coord | 20004 |
| DN dn001 | tbase01 | /data/tbase/pgxc/nodes/gtm | 20008 |
| DN dn002 | tbase02 | /data/tbase/pgxc/nodes/gtm | 20008 |
编写docker-compose.yml
version: '3.6'
networks:
inner:
name: tbase-cluster
driver: bridge
ipam:
config:
- subnet: 172.19.0.0/16
gateway: 172.19.0.1
services:
tbase01:
image: registry.cn-hangzhou.aliyuncs.com/snake-group/tbase-base:1.0
container_name: tbase01
# restart: always
privileged: true
shm_size: '256m'
ports:
- "20004:20004"
volumes:
- D:/dockerVolumes/tbase/tbase01/nodes:/data/tbase/pgxc/nodes
- D:/dockerVolumes/tbase/tbase01/pgxc_ctl:/data/tbase/pgxc_ctl
networks:
inner:
ipv4_address: 172.19.0.2
tbase02:
image: registry.cn-hangzhou.aliyuncs.com/snake-group/tbase-base:1.0
container_name: tbase02
# restart: always
privileged: true
shm_size: '256m'
volumes:
- D:/dockerVolumes/tbase/tbase02/nodes:/data/tbase/pgxc/nodes
networks:
inner:
ipv4_address: 172.19.0.3
本文中此文件入在:
D:\IdeaWorkspace\snake-group\document\docker\tbase\docker-compose.yml
因为容器启动默认root用户，所以导致容器内目录/data/tbase/pgxc/nodes权限为root,使用命令chown -R tbase:tbase /data/tbase/pgxcchown -R tbase:tbase /data/tbase/pgxc_ctl变更权限
启动容器节点tbase01, tbase02
docker-compose -p tbase-cluster -f D:\IdeaWorkspace\snake-group\document\docker\tbase\docker-compose.yml up -d
执行结果如下
PS D:\> docker-compose -p tbase-cluster -f D:\IdeaWorkspace\snake-group\document\docker\tbase\docker-compose.yml up -d
[+] Running 2/2
✔ Container tbase01 Started 1.0s
✔ Container tbase02 Started 1.0s
关于管理及查看Docker 建议使用Idea 并安装Docker插件查看
也可以用docker ps查看
建立tbase01,tbase02 SSH互信
分别进入容器节点执行以下操作(所有操作以tbase用户执行 su tbase)
# 生成密钥对一路回车就好
ssh-keygen -t rsa
结果如下
[tbase@0a86faf6c133 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/data/tbase/.ssh/id_rsa):
Created directory '/data/tbase/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /data/tbase/.ssh/id_rsa.
Your public key has been saved in /data/tbase/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:8Ktg/iyX9b4ndtQ1W0q+T5n8iID6lo7Rru+Sy5wnNT8 tbase@0a86faf6c133
The key's randomart image is:
+---[RSA 2048]----+
| |
| |
| . |
| o ..o|
| S + o+|
| .+o . =.o|
| o .=+=.. =.|
| o.=*B+ E.o..o.|
| .+%%*oo*. ..o|
+----[SHA256]-----+
再执行
ssh-copy-id -i /data/tbase/.ssh/id_rsa.pub tbase@tbase01
ssh-copy-id -i /data/tbase/.ssh/id_rsa.pub tbase@tbase02
此使命令执行行会让你输入密码，如果按照本文件操作密码应该是
tbase
互信检测
在tbase01执行下面命令，不输入密码则为成功
ssh tbase@tbase02
在tbase02执行下面命令，不输入密码则为成功
ssh tbase@tbase01
在tbase01节点生成pgxc_ctl.conf配置文件(tbase用户)
pgxc_ctl prepare config
结果如下
[tbase@dd91afb75cd3 .ssh]$ pgxc_ctl prepare config
/usr/bin/bash
Installing pgxc_ctl_bash script as /data/tbase/pgxc_ctl/pgxc_ctl_bash.
ERROR: File "/data/tbase/pgxc_ctl/pgxc_ctl.conf" not found or not a regular file. No such file or directory
Installing pgxc_ctl_bash script as /data/tbase/pgxc_ctl/pgxc_ctl_bash.
Reading configuration using /data/tbase/pgxc_ctl/pgxc_ctl_bash --home /data/tbase/pgxc_ctl --configuration /data/tbase/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
******** PGXC_CTL START ***************
Current directory: /data/tbase/pgxc_ctl
查看目录及文件
ll /data/tbase/pgxc_ctl
配置tbase01节点的pgxc_ctl.conf文件(tbase用户)
#!/usr/bin/env bash
srv1=tbase01
srv2=tbase02
pgxcInstallDir=$HOME/tbase_bin_v2.4
pgxcOwner=$USER
pgxcUser=$pgxcOwner
defaultDatabase=postgres
pgxcMainClusterName=cluster1
tmpDir=/tmp
localTmpDir=$tmpDir
configBackup=n
configBackupHost=pgxc-linker
configBackupDir=$HOME/pgxc
configBackupFile=pgxc_ctl.bak
#---- GTM ------------------------------------------------------------------------------------
gtmName=gtm
gtmMasterServer=$srv1
gtmMasterPort=20001
gtmMasterDir=$HOME/pgxc/nodes/gtm
gtmExtraConfig=none
gtmMasterSpecificExtraConfig=none
gtmSlave=y
gtmSlaveName=gtmSlave
gtmSlaveServer=$srv2
gtmSlavePort=20001
gtmSlaveDir=$HOME/pgxc/nodes/gtm
gtmSlaveSpecificExtraConfig=none
#---- Coordinators ----------------------------------------------------------------------------------------------------
coordMasterDir=$HOME/pgxc/nodes/coord
coordSlaveDir=$HOME/pgxc/nodes/coord_slave
coordArchLogDir=$HOME/pgxc/nodes/coord_archlog
coordNames=(cn001 cn002)
coordPorts=(20004 20004)
poolerPorts=(20010 20010)
coordPgHbaEntries=(0.0.0.0/0)
coordMasterServers=($srv1 $srv2)
coordMasterDirs=($coordMasterDir $coordMasterDir)
coordMaxWALsernder=2
coordMaxWALSenders=($coordMaxWALsernder $coordMaxWALsernder)
coordSlave=n
coordSlaveSync=n
coordArchLogDirs=($coordArchLogDir $coordArchLogDir)
coordExtraConfig=coordExtraConfig
cat > $coordExtraConfig <<EOF
#================================================
# Added to all the coordinator postgresql.conf
# Original: $coordExtraConfig
# 此处适应性修改后 dn slave可正常启动
#listen_addresses = '0.0.0.0'
max_connections = 500
max_pool_size = 1000
shared_buffers = 1GB
max_prepared_transactions = 2000
maintenance_work_mem = 256MB
wal_level = logical
max_wal_senders = 64
max_wal_size = 1GB
min_wal_size = 256MB
log_destination = 'csvlog'
logging_collector = on
log_directory = 'pg_log'
log_filename = 'postgresql-%A-%H.log'
synchronous_commit = local
synchronous_standby_names = ''
EOF
coordSpecificExtraConfig=(none none)
coordExtraPgHba=coordExtraPgHba
cat > $coordExtraPgHba <<EOF
local all all trust
host all all 0.0.0.0/0 trust
host replication all 0.0.0.0/0 trust
host all all ::1/128 trust
host replication all ::1/128 trust
host replication all $srv1 trust
host all all $srv1 trust
host replication all $srv2 trust
host all all $srv2 trust
host all all 0.0.0.0/0 md5
EOF
coordSpecificExtraPgHba=(none none)
coordAdditionalSlaves=n
cad1_Sync=n
#---- Datanodes -------------------------------------------------------------------------------------------------------
datanodeMasterDir1=$HOME/pgxc/nodes/dn_master1
datanodeMasterDir2=$HOME/pgxc/nodes/dn_master2
datanodeSlaveDir1=$HOME/pgxc/nodes/dn_slave1
datanodeSlaveDir2=$HOME/pgxc/nodes/dn_slave2
datanodeArchLogDir=$HOME/pgxc/nodes/datanode_archlog
primaryDatanode=dn001
datanodeNames=(dn001 dn002)
datanodePorts=(20008 20008)
datanodePoolerPorts=(20018 20018)
datanodePgHbaEntries=(0.0.0.0/0)
datanodeMasterServers=($srv1 $srv2)
datanodeMasterDirs=($datanodeMasterDir1 $datanodeMasterDir2)
datanodeMaxWalSender=4
datanodeMaxWALSenders=($datanodeMaxWalSender $datanodeMaxWalSender)
datanodeSlave=y
datanodeSlaveServers=($srv2 $srv1)
datanodeSlavePorts=(20009 20009)
datanodeSlavePoolerPorts=(20019 20019)
datanodeSlaveSync=n
datanodeSlaveDirs=($datanodeSlaveDir1 $datanodeSlaveDir2)
datanodeArchLogDirs=($datanodeArchLogDir/dn_master1 $datanodeArchLogDir/dn_master2)
datanodeExtraConfig=datanodeExtraConfig
cat > $datanodeExtraConfig <<EOF
#================================================
# Added to all the coordinator postgresql.conf
# Original: $datanodeExtraConfig
# 此处适应性修改后 dn slave可正常启动
#listen_addresses = '0.0.0.0'
max_connections = 500
max_pool_size = 1000
shared_buffers = 1GB
max_prepared_transactions = 2000
maintenance_work_mem = 256MB
wal_level = logical
max_wal_senders = 64
max_wal_size = 1GB
min_wal_size = 256MB
log_destination = 'csvlog'
logging_collector = on
log_directory = 'pg_log'
log_filename = 'postgresql-%A-%H.log'
synchronous_commit = local
synchronous_standby_names = ''
EOF
datanodeSpecificExtraConfig=(none none)
datanodeExtraPgHba=datanodeExtraPgHba
cat > $datanodeExtraPgHba <<EOF
local all all trust
host all all 0.0.0.0/0 trust
host replication all 0.0.0.0/0 trust
host all all ::1/128 trust
host replication all ::1/128 trust
host replication all $srv1 trust
host all all $srv1 trust
host replication all $srv2 trust
host all all $srv2 trust
host all all 0.0.0.0/0 md5
EOF
datanodeSpecificExtraPgHba=(none none)
datanodeAdditionalSlaves=n
walArchive=n
本文件位于:
/data/tbase/pgxc_ctl/pgxc_ctl.conf
用pgxc_ctl初始化集群(tbase用户)
进入pgxc_ctl 管理工具
pgxc_ctl
结果如下
[tbase@dd91afb75cd3 pgxc_ctl]$ pgxc_ctl
/usr/bin/bash
Installing pgxc_ctl_bash script as /data/tbase/pgxc_ctl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /data/tbase/pgxc_ctl/pgxc_ctl_bash.
Reading configuration using /data/tbase/pgxc_ctl/pgxc_ctl_bash --home /data/tbase/pgxc_ctl --configuration /data/tbase/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
******** PGXC_CTL START ***************
Current directory: /data/tbase/pgxc_ctl
PGXC
分发二进制工具包
deploy all
结果如下
PGXC deploy all
Deploying Postgres-XL components to all the target servers.
Prepare tarball to deploy ...
Deploying to the server tbase01.
Deploying to the server tbase02.
Deployment done.
初始化集群
init all
执行结果如下
PGXC init all
Initialize GTM master
The files belonging to this GTM system will be owned by user "tbase".
This user must also own the server process.
......
ALTER NODE cn001 WITH (HOST='tbase01', PORT=20004);
ALTER NODE
CREATE NODE cn002 WITH (TYPE='coordinator', HOST='tbase02', PORT=20004);
CREATE NODE
CREATE NODE dn001 WITH (TYPE='datanode', HOST='tbase01', PORT=20008, PRIMARY, PREFERRED);
CREATE NODE
CREATE NODE dn002 WITH (TYPE='datanode', HOST='tbase02', PORT=20008);
CREATE NODE
SELECT pgxc_pool_reload();
pgxc_pool_reload
------------------
t
(1 row)
CREATE NODE cn001 WITH (TYPE='coordinator', HOST='tbase01', PORT=20004);
CREATE NODE
ALTER NODE cn002 WITH (HOST='tbase02', PORT=20004);
ALTER NODE
CREATE NODE dn001 WITH (TYPE='datanode', HOST='tbase01', PORT=20008, PRIMARY);
CREATE NODE
CREATE NODE dn002 WITH (TYPE='datanode', HOST='tbase02', PORT=20008, PREFERRED);
CREATE NODE
SELECT pgxc_pool_reload();
pgxc_pool_reload
------------------
t
(1 row)
Done.
EXECUTE DIRECT ON (dn001) 'CREATE NODE cn001 WITH (TYPE=''coordinator'', HOST=''tbase01'', PORT=20004)';
EXECUTE DIRECT
EXECUTE DIRECT ON (dn001) 'CREATE NODE cn002 WITH (TYPE=''coordinator'', HOST=''tbase02'', PORT=20004)';
EXECUTE DIRECT
EXECUTE DIRECT ON (dn001) 'ALTER NODE dn001 WITH (TYPE=''datanode'', HOST=''tbase01'', PORT=20008, PRIMARY, PREFERRED)';
EXECUTE DIRECT
EXECUTE DIRECT ON (dn001) 'CREATE NODE dn002 WITH (TYPE=''datanode'', HOST=''tbase02'', PORT=20008, PREFERRED)';
EXECUTE DIRECT
EXECUTE DIRECT ON (dn001) 'SELECT pgxc_pool_reload()';
pgxc_pool_reload
------------------
t
(1 row)
EXECUTE DIRECT ON (dn002) 'CREATE NODE cn001 WITH (TYPE=''coordinator'', HOST=''tbase01'', PORT=20004)';
EXECUTE DIRECT
EXECUTE DIRECT ON (dn002) 'CREATE NODE cn002 WITH (TYPE=''coordinator'', HOST=''tbase02'', PORT=20004)';
EXECUTE DIRECT
EXECUTE DIRECT ON (dn002) 'CREATE NODE dn001 WITH (TYPE=''datanode'', HOST=''tbase01'', PORT=20008, PRIMARY, PREFERRED)';
EXECUTE DIRECT
EXECUTE DIRECT ON (dn002) 'ALTER NODE dn002 WITH (TYPE=''datanode'', HOST=''tbase02'', PORT=20008, PREFERRED)';
EXECUTE DIRECT
EXECUTE DIRECT ON (dn002) 'SELECT pgxc_pool_reload()';
pgxc_pool_reload
------------------
t
(1 row)
Done.
查看服务状态
monitor all
结果如下
PGXC monitor all
Running: gtm master
Running: gtm slave
Running: coordinator master cn001
Running: coordinator master cn002
Running: datanode master dn001
Running: datanode slave dn001
Running: datanode master dn002
Running: datanode slave dn002
访问tbase集群(tbase用户)
从CN节点访问集群
psql -h tbase01 -p 20004 -d postgres -U tbase
结果如下
[tbase@dd91afb75cd3 ~]$ psql -h tbase02 -p 20004 -d postgres -U tbase
psql (PostgreSQL 10.0 TBase V2)
Type "help" for help.
postgres=#
退出psql用
\q
切换到tbase用\c
查看集群节点
select * from pgxc_node;
结果如下
postgres=# select * from pgxc_node;
node_name | node_type | node_port | node_host | nodeis_primary | nodeis_preferred | node_id | node_cluster_name
-----------+-----------+-----------+-----------+----------------+------------------+------------+-------------------
gtm | G | 20001 | tbase01 | t | f | 428125959 | tbase_cluster
cn001 | C | 20004 | tbase01 | f | f | -264077367 | tbase_cluster
cn002 | C | 20004 | tbase02 | f | f | -674870440 | tbase_cluster
dn001 | D | 20008 | tbase01 | t | t | 2142761564 | tbase_cluster
dn002 | D | 20008 | tbase02 | f | f | -17499968 | tbase_cluster
(5 rows)
创建default_group和sharding_group
使用数据库前必有创建分组
create default node group default_group with (dn001,dn002);
create sharding group to group default_group;
结果如下
postgres=# create default node group default_group with (dn001,dn002);
CREATE NODE GROUP
postgres=# create sharding group to group default_group;
CREATE SHARDING GROUP
现在我们可以像访问单例数据库一样访问数据库集群了
创建数据及用户
create database test;
create user test with password 'test';
alter database test owner to test;
结果如下
postgres=# create database test;
CREATE DATABASE
postgres=# create user test with password 'test';
CREATE ROLE
postgres=# alter database test owner to test;
ALTER DATABASE
postgres=#
连接到数据库test,用test用户
\c test test
结果如下
postgres=# \c test test
You are now connected to database "test" as user "test".
test=>
pspl 命令查看用
help指定test=> help You are using psql, the command-line interface to PostgreSQL. Type: \copyright for distribution terms \h for help with SQL commands \? for help with psql commands \g or terminate with semicolon to execute query \q to quit数据连接信息为: docker宿主机IP:20004
用户/密码: test/test
创建测试表并添加数据
create table foo(id bigint,name text) distribute by shard(id);
insert into foo(id,name) values(1, 'tecent'),(2,'alibaba');
select * from foo;
结果如下
test=> create table foo(id bigint,name text) distribute by shard(id);
CREATE TABLE
test=> insert into foo(id,name) values(1, 'tecent'),(2,'alibaba');
COPY 2
test=> select * from foo;
id | name
----+---------
1 | tecent
2 | alibaba
(2 rows)
test=>
tbase集群管理
pgxc_ctl
结果如下
[tbase@1f3c1078b9a2 app]$ pgxc_ctl
/usr/bin/bash
Installing pgxc_ctl_bash script as /data/tbase/pgxc_ctl/pgxc_ctl_bash.
ERROR: File "/data/tbase/pgxc_ctl/pgxc_ctl.conf" not found or not a regular file. No such file or directory
Installing pgxc_ctl_bash script as /data/tbase/pgxc_ctl/pgxc_ctl_bash.
Reading configuration using /data/tbase/pgxc_ctl/pgxc_ctl_bash --home /data/tbase/pgxc_ctl --configuration /data/tbase/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
******** PGXC_CTL START ***************
Current directory: /data/tbase/pgxc_ctl
PGXC
停止集群
stop all -m fast
monitor all
结果如下
PGXC stop all -m fast
Stopping all the coordinator masters.
Stopping coordinator master cn001.
Stopping coordinator master cn002.
Done.
Stopping all the datanode masters.
Stopping datanode master dn001.
Stopping datanode master dn002.
Done.
Stop GTM slave
waiting for server to shut down.... done
server stopped
Stop GTM master
waiting for server to shut down.... done
server stopped
PGXC
启动集群
start all
monitro all
结果如下
PGXC start all
Start GTM master
server starting
..........
Start GTM slaveserver starting
..........
Starting coordinator master.
Starting coordinator master cn001
Starting coordinator master cn002
..........
Done.
Starting all the datanode masters.
Starting datanode master dn001.
Starting datanode master dn002.
..........
Done.
PGXC monitor all
Running: gtm master
Running: gtm slave
Running: coordinator master cn001
Running: coordinator master cn002
Running: datanode master dn001
Running: datanode master dn002
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
