minio多节点单驱动器（多驱动器一样配置）

最新推荐文章于 2025-04-02 07:51:43 发布

热心市民运维小孙

最新推荐文章于 2025-04-02 07:51:43 发布

阅读量708

点赞数 10

分类专栏： minio 分布式对象存储服务器 linux运维文章标签： linux 服务器运维

本文链接：https://blog.youkuaiyun.com/weixin_67274256/article/details/143432473

版权

linux运维同时被 3 个专栏收录

16 篇文章

订阅专栏

minio

1 篇文章

订阅专栏

分布式对象存储服务器

1 篇文章

订阅专栏

miniO 集群部署搭建（4节点单驱动器）

准备工作

关防火墙或者放行9001、9000（默认）

开放永久开放防火墙端口：firewall-cmd --zone=public --add-port=9000/tcp --permanentsuccess

重启防火墙：firewall-cmd --reload

检查：firewall-cmd --zone=public --list-port

挂载磁盘
1. fdisk -l 或者lsblk 看那个磁盘
2. mkfs.xfs /dev/sdb -L DISK1 //格式化磁盘为xfs格式
3. 创建文件夹为挂载源 mkdir /minio1 （多驱动器就多个文件夹挂载到对应驱动器）
4. mount挂载：mount /dev/sdb /minio1
5. df -h 检查挂载是否成功
6. vim /etc/fstab 修改这个文件来到达开机自动挂载的目的
7. 文件添加:/dev/sdb /minio1 xfs defaults,noatime 0 2 (noatime是访问文件时不更新文件的访问时间 0 2 的2是表示这个文件系统将在根文件系统之后较早地进行检查，文件系统检查的顺序 /是1，确保文件完整性和一致性)

centos分区方法 利用fdisk
fdisk /dev/sda
1.进入fdisk命令行界面后，会看到类似下面的提示符
   Welcome to fdisk (util - linux 2.37.4).
   Changes will remain in memory only, until you decide to write them.
   Be careful before using the write command.
   Command (m for help): 
2.查看帮助信息
在fdisk命令提示符下输入m可以查看帮助信息。它会列出所有可用的命令，如n（新建分区）、d（删除分区）、p（打印分区表）、w（将分区表写入磁盘并退出）、q（不保存分区表修改直接退出）等。
3.打印分区表
输入p命令可以打印当前磁盘的分区表
4.创建新分区
输入n命令可以创建一个新分区。然后fdisk会询问是创建主分区（p）还是扩展分区（e）。对于大多数情况，如果是 MBR 分区表，最多可以创建 4 个主分区或者 3 个主分区和 1 个扩展分区。例如，创建一个主分区：p

4.1
选择p后，它会要求输入分区号、起始扇区和结束扇区等信息。如果直接按回车键，fdisk会自动分配默认值。例如：
   Partition number (3,4, default 3): 
   First sector (209713152 - 209715200, default 209713152): 
   Last sector, +sectors or +size{K,M,G,T,P} (209713152 - 209715200, default 209715200): +1G
   
5.删除分区
   输入d命令可以删除一个分区。它会要求输入要删除的分区号。例如:
      Command (m for help): d
   Partition number (1 - 3): 3
6.保存分区表修改并退出
完成分区操作后，如果要将修改后的分区表写入磁盘，输入w命令。例如
   Command (m for help): w
   The partition table has been altered.
   Calling ioctl() to re - read partition table.
   Syncing disks.
不想修改按q退出 GPT分区需要用到别的工具fdisk功能有限

安装ntp同步时间或者date检查一下

搭建（核心）

下载

wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio-20241013133411.0.0-1.x86_64.rpm -O minio.rpm

-O minio.rpm  是将下载的这个东西命名为 minio.rpm保存

//离线操作直接下载二进制文件安装

下载完成后安装

rpm -i minio.rpm

文件安装的默认位置：/usr/local/bin
默认会产生 /usr/lib/systemd/system/minio.service

配置

这一步是为了让我们后面可以直接使用systemctl start minio这样的方式启动

vim /etc/systemd/system/minio.service

minio.servie：

[Unit]
Description=MinIO
Documentation=https://docs.min.io
Wants=network-online.target
After=network-online.target
#确保他是可执行文件位置
AssertFileIsExecutable=/usr/local/bin/minio    

[Service]
#下面的路径对就行
WorkingDirectory=/usr/local

User=minio-user
Group=minio-user
ProtectProc=invisible

EnvironmentFile=-/etc/default/minio
ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi"
ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES

# Let systemd restart this service always
Restart=always

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=1048576

# Specifies the maximum number of threads this process can create
TasksMax=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

# Built for ${project.name}-${project.version} (${project.name})

创建用户：

groupadd -r minio-user
useradd -M -r -g minio-user minio-user
chown minio-user:minio-user /minio1 
-r系统组 id500以内
-M不自动建立用户的登入目录 -r系统账号 -g 所属组
修改所属组

修改环境变量文件

vim /etc/default/minio // 最主要的文件，报错原因大部分就是这里没配好

官网的：
# Set the hosts and volumes MinIO uses at startup
# The command uses MinIO expansion notation {x...y} to denote a
# sequential series.
#
# The following example covers four MinIO hosts
# with 4 drives each at the specified hostname and drive locations.
# The command includes the port that each MinIO server listens on
# (default 9000)

MINIO_VOLUMES="https://minio{1...4}.example.net:9000/mnt/disk{1...4}/minio"

# Set all MinIO server options
#
# The following explicitly sets the MinIO Console listen address to
# port 9001 on all network interfaces. The default behavior is dynamic
# port selection.

MINIO_OPTS="--console-address :9001"

# Set the root username. This user has unrestricted permissions to
# perform S3 and administrative API operations on any resource in the
# deployment.
#
# Defer to your organizations requirements for superadmin user name.

MINIO_ROOT_USER=minioadmin

# Set the root password
#
# Use a long, random, unique string that meets your organizations
# requirements for passwords.

MINIO_ROOT_PASSWORD=minio-secret-key-CHANGE-ME

# Set to the URL of the load balancer for the MinIO deployment
# This value *must* match across all MinIO servers. If you do
# not have a load balancer, set this value to to any *one* of the
# MinIO hosts in the deployment as a temporary measure.
MINIO_SERVER_URL="https://minio.example.net:9000"

自己配置的：

# Set the hosts and volumes MinIO uses at startup
# The command uses MinIO expansion notation {x...y} to denote a
# sequential series.
#
# The following example covers four MinIO hosts
# with 4 drives each at the specified hostname and drive locations.
# The command includes the port that each MinIO server listens on
# (default 9000)
## 这块是文件磁盘的位置 因为我们是集群节点是
MINIO_VOLUMES="http://192.168.200.88:9000/minio1 http://192.168.200.34:9000/minio1 http://192.168.200.53:9000/minio1 http://192.168.200.118:9000/minio1"

# Set all MinIO server options
#
# The following explicitly sets the MinIO Console listen address to
# port 9001 on all network interfaces. The default behavior is dynamic
# port selection.

## minio-console的地址 就是web界面控制台
MINIO_OPTS="--console-address :9001"

# Set the root username. This user has unrestricted permissions to
# perform S3 and administrative API operations on any resource in the
# deployment.
#
# Defer to your organizations requirements for superadmin user name.

# console的登陆账号
MINIO_ROOT_USER=minioadmin

# Set the root password
#
# Use a long, random, unique string that meets your organizations
# requirements for passwords.

# console的登陆密码
MINIO_ROOT_PASSWORD=minioadmin

# Set to the URL of the load balancer for the MinIO deployment
# This value *must* match across all MinIO servers. If you do
# not have a load balancer, set this value to to any *one* of the
# MinIO hosts in the deployment as a temporary measure.
# 负载均衡地址 如果没有负载均衡就不要放开这个配置
#MINIO_SERVER_URL="http://192.168.200.46:9000"

还有一种池化配置，最好连续的ip或者域名

不连续：MINIO_VOLUMES="http://192.168.200.118:9000/minio1http://192.168.200.155:9000/minio1"

连续-池化写法：MINIO_VOLUMES="http://192.168.200.16{3...6}:9000/minio{1...4}"

启动

systemctl start minio

访问节点登进去

账号minioadmin

密码minioadmin

在监控栏看server是否是4 drive 是否是驱动器个数

不吹不黑，至此通关

负载均衡

安装nginx省略

主要修改nginx.conf就行

worker_processes  1;

events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    
    keepalive_timeout  65;

    upstream minio_server {
			server	192.168.200.88:9000;
			server  192.168.200.34:9000;
			server  192.168.200.53:9000;
			server  192.168.200.118:9000;
    }

    upstream minio_console {
			server 192.168.200.88:9001;
			server 192.168.200.34:9001;
			server 192.168.200.53:9001;
			server 192.168.200.118:9001;
    }


    server {
        listen       9000;
        server_name  localhost;

        location / {
            proxy_pass http://minio_server;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
  	server {
        listen       9001;
        server_name  localhost;

        location / {
            proxy_pass http://minio_console;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

    }
}

验证：访问nginx代理的地址:192.168.200.46:9001

测试

创建桶正常

upload正常

假设节点为N，宕机一半仍可读，宕机一半+1可读可写

问题：nginx代理负载均衡之后，访问桶文件一直加载，看不了桶的文件，访问单个minio服务器可以看桶文件

还有我没看懂这个实际存储空间咋计算的。