Centos安装FastDFS集群文件系统_fastdfs集群安装-优快云博客

本文链接：https://blog.youkuaiyun.com/u011385445/article/details/77480757

本文详细介绍了在Centos 7.3环境下，如何安装配置FastDFS集群，包括tracker和storage服务器的设置，文件上传下载流程，以及结合nginx、php-fpm实现文件访问。FastDFS是一个轻量级分布式文件系统，适用于中小文件存储服务。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

简介

FastDFS是一个开源的轻量级分布式文件系统。它解决了大数据量存储和负载均衡等问题。特别适合以中小文件（建议范围：4KB < file_size <500MB）为载体的在线服务，如相册网站、视频网站等等。在UC基于FastDFS开发向用户提供了：网盘，社区，广告和应用下载等业务的存储服务。

架构

FastDFS服务端有三个角色：跟踪服务器（tracker server）、存储服务器（storage server）和客户端（client）。

tracker server：跟踪服务器，主要做调度工作，起负载均衡的作用。在内存中记录集群中所有存储组和存储服务器的状态信息，是客户端和数据服务器交互的枢纽。相比GFS中的master更为精简，不记录文件索引信息，占用的内存量很少。Tracker cluster中各个tracker server相互独立，不进行相互通信。
storage server：存储服务器（又称：存储节点或数据服务器），文件和文件属性（meta data）都保存到存储服务器上。Storage server直接利用OS的文件系统调用管理文件。Storage cluster中各个storage组（Volume1,Volume2…）相互独立，不进行相互通信，也就是说各个组之间保存的数据是不相同的。但是各个组中的storage server之间是属于互相备份的关系，也就是说storage server之间保存相同的数据。每个storage server会启动一个单独的线程主动向Tracker cluster中每个tracker server报告其状态信息，包括磁盘使用情况，文件同步情况及文件上传下载次数统计等信息。
client：客户端，作为业务请求的发起方，通过专有接口，使用TCP/IP协议与跟踪器服务器或存储节点进行数据交互。

这里写图片描述

工作流程

文件上传

1）Client通过Tracker server将文件上传到Storage server。
2）Tracker server向Client返回一台可用的Storage server的IP地址和端口号。
3）Client直接通过Tracker server返回的IP地址和端口与其中一台Storage server建立连接并进行文件上传。
4）上传完成，Storage server返回给Client上传的文件名和文件组，文件上传结束。
文件下载

1）Client通过Tracker server下载指定Storage组中某个Storage server上的某个文件（文件名包括Storage组名称）。
2）Tracker server向Client返回一台可用的Storage server的IP地址和端口号。
3）Client直接通过Tracker server返回的IP地址和端口与其中一台Storage server建立连接并进行文件下载。

安装配置集群

一、系统环境

CentOS 7.3 64位
FastDFS_v5.08.tar.gz
nginx-1.8.1.tar.gz
fastdfs-nginx-module_v1.16.tar.gz
libfastcommon

二、服务器规划

tracker server：172.18.10.10:22122
storage server 1：172.18.10.11:23000
storage server 2:172.18.10.12:23000
2台storage server 互为备份，同属一个group。
所有服务器按照以下步骤安装，启动时分别启动对应的tracker或storage即可。
tracker服务器中可以安装使用php或者java驱动，上传文件。
nginx安装目录：/application/nginx/
FastDFS配置文件目录：/etc/fdfs/
FastDFS脚本服务文件目录：/etc/init.d/
上传文件存放目录：/data/fastdfs/
安装程序存放目录：/home/fdfs/tools/
配置ip地址可以填写内网地址
外网访问需要开通相应访问端口

三、安装FastDFS

1、创建并进入程序存放目录

[root@fastdfs-storage ~]# mkdir -p /home/fdfs/tools
[root@fastdfs-storage ~]# cd /home/fdfs/tools/

2、下载并安装FastDFS依赖包 libfastcommon

[root@fastdfs-storage tools]# wget https://codeload.github.com/happyfish100/libfastcommon/zip/master
[root@fastdfs-storage tools]# unzip master
[root@fastdfs-storage tools]# cd libfastcommon-master/
[root@fastdfs-storage libfastcommon-master]# ls
HISTORY  INSTALL  libfastcommon.spec  make.sh  php-fastcommon  README  src
[root@fastdfs-storage libfastcommon-master]# ./make.sh
[root@fastdfs-storage libfastcommon-master]# ./make.sh install

3、下载并安装FastDFS

[root@fastdfs-tracker tools]# wget https://codeload.github.com/happyfish100/fastdfs/zip/V5.11
[root@fastdfs-tracker tools]# unzip V5.11
[root@fastdfs-tracker tools]# cd fastdfs-5.11
[root@fastdfs-tracker FastDFS]# ./make.sh 
[root@fastdfs-tracker FastDFS]# ./make.sh install

ps：多tracker情况下，5.10版本以上的FastDFS才支持。

默认方式安装完成之后，文件及目录：

服务脚本：

[root@fastdfs-tracker ~]# ll /etc/init.d/ |grep fdfs
-rwxr-xr-x. 1 root root   918 4月  22 22:08 fdfs_storaged
-rwxr-xr-x. 1 root root   920 4月  22 22:08 fdfs_trackerd

配置文件

[root@fastdfs-tracker ~]# ll /etc/fdfs/
total 20
-rw-r--r--. 1 root root 1461 4月  22 22:08 client.conf.sample
-rw-r--r--. 1 root root 7927 4月  22 22:08 storage.conf.sample
-rw-r--r--. 1 root root 7200 4月  22 22:08 tracker.conf.sample

命令行工具

[root@fastdfs-tracker ~]# ll /usr/bin|grep fdfs
-rwxr-xr-x  1 root root     315496 Aug 19 17:07 fdfs_appender_test
-rwxr-xr-x  1 root root     315272 Aug 19 17:07 fdfs_appender_test1
-rwxr-xr-x  1 root root     302120 Aug 19 17:07 fdfs_append_file
-rwxr-xr-x  1 root root     301800 Aug 19 17:07 fdfs_crc32
-rwxr-xr-x  1 root root     302184 Aug 19 17:07 fdfs_delete_file
-rwxr-xr-x  1 root root     302920 Aug 19 17:07 fdfs_download_file
-rwxr-xr-x  1 root root     302504 Aug 19 17:07 fdfs_file_info
-rwxr-xr-x  1 root root     316336 Aug 19 17:07 fdfs_monitor
-rwxr-xr-x  1 root root    1102216 Aug 19 17:07 fdfs_storaged
-rwxr-xr-x  1 root root     325432 Aug 19 17:07 fdfs_test
-rwxr-xr-x  1 root root     320552 Aug 19 17:07 fdfs_test1
-rwxr-xr-x  1 root root     448032 Aug 19 17:07 fdfs_trackerd
-rwxr-xr-x  1 root root     303112 Aug 19 17:07 fdfs_upload_appender
-rwxr-xr-x  1 root root     304128 Aug 19 17:07 fdfs_upload_file

注意：虽然FastDFS区分tracker和storage服务器，但是安装的软件及步骤均相同，只是不同的配置文件而已，因此以上安装适用tracker server和storage server。
4、配置tracker server

拷贝tracker server和client端配置文件并重命名

[root@fastdfs-tracker ~]# cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf
[root@fastdfs-storage ~]# cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf

编辑tracker server配置文件tracker.conf，修改内容如下：

    disabled=false（默认为false，表示是否无效）
    port=22122（默认为22122）
    base_path=/data/fastdfs/tracker

创建tracker server数据目录

[root@fastdfs-tracker ~]# mkdir -p /data/fastdfs/tracker

配置client配置文件client.conf，修改内容如下：

ps：php或java程序驱动通过调用该client配置文件，上传文件

    base_path=/data/fastdfs/tracker
    tracker_server=172.18.10.10:22122

测试启动tracker server，启动成功会自动在/data/fastdfs/tracker目录新建data和logs目录

[root@fastdfs-tracker ~]# cd /data/fastdfs/tracker/
[root@fastdfs-tracker tracker]# ls
[root@fastdfs-tracker tracker]# /etc/init.d/fdfs_trackerd start
Starting FastDFS tracker server:
[root@fastdfs-tracker tracker]# ss -lntup|grep 22122
tcp    LISTEN     0      128                    *:22122                 *:*      users:(("fdfs_trackerd",3785,5)) 
[root@fastdfs-tracker tracker]# ls
data  logs

关闭tracker server命令

[root@fastdfs-tracker tracker]# /etc/init.d/fdfs_trackerd stop

5、配置storage server

拷贝storage server配置文件并重命名

[root@fastdfs-storage ~]# cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf

编辑storage server配置文件storage.conf，修改内容如下：

    disabled=false（默认为false，表示是否无效）
    port=23000（默认为23000）
    base_path=/data/fastdfs/storage
    tracker_server=172.18.10.10:22122
    store_path0=/data/fastdfs/storage
    http.server_port=8888（默认为8888，nginx中配置的监听端口那之一致）

ps：两台storage server配置相同

创建storage server数据目录

[root@fastdfs-storage ~]# mkdir -p /data/fastdfs/storage

测试启动storage server，启动成功会自动在/data/fastdfs/tracker目录新建data和logs目录，启动storage server时，tracker server已启动

[root@fastdfs-storage ~]# cd /data/fastdfs/storage/
[root@fastdfs-storage storage]# ls
[root@fastdfs-storage storage]# /etc/init.d/fdfs_storaged start
Starting FastDFS storage server:
[root@fastdfs-storage storage]# ss -lntup|grep 23000
tcp    LISTEN     0      128                    *:23000                 *:*      users:(("fdfs_storaged",3786,5))
[root@fastdfs-storage storage]# ls
data  logs

6、测试文件上传
在tracker server中，执行以下命令，并在/home/fdfs/image/下存在test.jpg图片

[root@fastdfs-tracker ~]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /home/fdfs/image/test.jpg
group1/M00/00/00/tBIK6VcaP0aARXasAAHrUgHEviQ394.jpg

返回文件名字，即说明文件上传成功。
7、存储服务器tracker server 安装并配置nginx

下载并安装fastdfs-nginx-module模块

注：FastDFS通过Tracker服务器,将文件放在Storage服务器存储，但是同组存储服务器之间需要进入文件复制，有同步延迟的问题。假设Tracker服务器将文件上传到了172.18.10.11，上传成功后文件ID已经返回给客户端。此时FastDFS存储集群机制会将这个文件同步到同组存储172.18.10.12，在文件还没有复制完成的情况下，客户端如果用这个文件ID在172.18.10.12上取文件,就会出现文件无法访问的错误。而fastdfs-nginx-module可以重定向文件连接到源服务器取文件,避免客户端由于复制延迟导致的文件无法访问错误。

进入/home/fdfs/tools文件夹

[root@fastdfs-tracker ~]# cd /home/fdfs/tools
[root@fastdfs-storage  tools]# wget http://nchc.dl.sourceforge.net/project/fastdfs/FastDFS%20Nginx%20Module%20Source%20Code/fastdfs-nginx-module_v1.16.tar.gz
[root@fastdfs-storage tools]# tar xf fastdfs-nginx-module_v1.16.tar.gz 
[root@fastdfs-storage tools]# cd fastdfs-nginx-module/src/

修改config文件，执行如下命令，进行批量替换，并保存退出

[root@fastdfs-storage src]# vim config
:%s+/usr/local/+/usr/+g

拷贝fastdfs-nginx-module模块中配置文件到/etc/fdfs目录中并编辑

[root@fastdfs-storage ~]# cp /home/fdfs/tools/fastdfs-nginx-module/src/mod_fastdfs.conf /etc/fdfs/
[root@fastdfs-storage ~]# vim /etc/fdfs/mod_fastdfs.conf

修改配置文件如下：

connect_timeout=10
base_path=/tmp（默认为/tmp）
tracker_server=172.18.10.10:22122
storage_server_port=23000（默认配置为23000）
url_have_group_name = true
store_path0=/data/fastdfs/storage
group_name=group1（默认配置为group1）

安装nginx依赖库

[root@fastdfs-storage nginx-1.8.1]# yum install -y pcre-devel zlib-devel

下载并安装nginx

[root@fastdfs-storage  tools]# wget http://nginx.org/download/nginx-1.8.1.tar.gz
[root@fastdfs-storage tools]# tar xf nginx-1.8.1.tar.gz 
[root@fastdfs-storage tools]# cd nginx-1.8.1
[root@fastdfs-storage nginx-1.8.1]# ./configure --prefix=/application/nginx/ --add-module=../fastdfs-nginx-module/src/
[root@fastdfs-storage nginx-1.8.1]# make && make install

如果报错缺少gcc，需要先安装gcc

[root@fastdfs-storage nginx-1.8.1]# yum install gcc gcc-c++

拷贝FastDFS中的部分配置文件到/etc/fdfs目录中

[root@fastdfs-storage ~]# cp /home/fdfs/tools/FastDFS/conf/http.conf /etc/fdfs/
[root@fastdfs-storage ~]# cp /home/fdfs/tools/FastDFS/conf/mime.types /etc/fdfs/

配置nginx，如下：

[root@fastdfs-storage ~]# vim /application/nginx/conf/nginx.conf
    user  root;    
    worker_processes  1;
    events {
       worker_connections  1024;
    }
    http {
        include       mime.types;
        default_type  application/octet-stream;
        sendfile        on;
        keepalive_timeout  65;
        server {
            listen       8888;
            server_name  www.baidu.com;
            location ~/group[0-9]/ {
                ngx_fastdfs_module;
            }
            error_page   500 502 503 504  /50x.html;
            location = /50x.html {
            root   html;
            }
        }
    }

主要配置server下的内容，接收请求到对应的文件。

a、"user root"是解决下载操作时报404的问题
b、8888端口号与/etc/fdfs/storage.conf中的http.server_port=8888相对应
c、storage对应有多个group的情况下，访问路径带group名称，例如：/group1/M00/00/00/**，对应nginx配置：
    location ~/group[0-9]/ {
           ngx_fastdfs_module;
     }

拷贝nginx服务到/etc/init.d/目录下，并启动nginx

[root@fastdfs-storage ~]# cp /application/nginx/sbin/nginx /etc/init.d/
[root@fastdfs-storage ~]# /etc/init.d/nginx

配置storage server中的nginx 配置文件中的server部分

server {
        listen       80;
        server_name  localhost;

        access_log  logs/fdfs.access.log;

        location ~/group[0-9]/ {
                ngx_fastdfs_module;
        }
    }

通过浏览器可以访问上传的文件

http://www.baidu.com:8888/group1/M00/00/00/tBIK6VcaP0aARXasAAHrUgHEviQ394.jpg

在2台storage server对应目录下可以检查是否有该文件存在

/data/fastdfs/storage/data/00/00/

8、使用php程序在tracker中做上传服务后台

安装php

yum install php
yum install php-devel

安装php-fpm依赖

yum install php-mysql php-mbstring

安装php-fpm

yum install php-fpm

安装FastDFS中php客户端上传支持

cd /home/fastdfs/tools/FastDFS/php_client/
phpize
./configure --with-php-config=/usr/bin/php-config
make
make install
cat /home/fastdfs/tools/FastDFS/php_client/fastdfs_client.ini >> /etc/php.ini

启动php-fpm

systemctl start php-fpm

配置nginx中支持php server

主要配置server内容

server {
    listen       80;
    server_name  www.google.com;
    index index.php;
    root /data/www/fdfsclient;

    access_log  logs/google.log;

    location / {
       if (!-e $request_filename) {
       rewrite  ^(.*)$  /index.php?s=/$1  last;
       break;
        }
     }

    location ~ .php($|/) {
        set $script $uri;
        set $path_info "";

        if ($uri ~ "^(.+.php)(/.+)") {
            set $script $1;
            set $path_info $2;
        }

        fastcgi_param SCRIPT_FILENAME $document_root$script;
        fastcgi_param SCRIPT_NAME $script;
        fastcgi_param PATH_INFO $path_info;

        try_files $uri =404;
        fastcgi_pass  127.0.0.1:9000;
        fastcgi_index index.php;
        include fastcgi.conf;
    }
}

附上FastDFS官方php操作方法


Copyright (C) 2008 Happy Fish / YuQing

FastDFS client php extension may be copied only under the terms of 
the Less GNU General Public License (LGPL).

Please visit the FastDFS Home Page for more detail.
Google code (English language): http://code.google.com/p/fastdfs/
Chinese language: http://www.csource.com/

In file fastdfs_client.ini, item fastdfs_client.tracker_group# point to
the FastDFS client config filename. Please read ../INSTALL file to know 
about how to config FastDFS client.

FastDFS client php extension compiled under PHP 5.2.x, Steps:
phpize
./configure
make
make install

#copy lib file to php extension directory, eg. /usr/lib/php/20060613/
cp modules/fastdfs_client.so  /usr/lib/php/20060613/

#copy fastdfs_client.ini to PHP etc directory, eg. /etc/php/
cp fastdfs_client.ini /etc/php/

#modify config file fastdfs_client.ini, such as:
vi /etc/php/fastdfs_client.ini

#run fastdfs_test.php
php fastdfs_test.php


FastDFS PHP functions:

string fastdfs_client_version()
return client library version


long fastdfs_get_last_error_no()
return last error no


string fastdfs_get_last_error_info()
return last error info


string fastdfs_http_gen_token(string remote_filename, int timestamp)
generate anti-steal token for HTTP download
parameters:
    remote_filename: the remote filename (do NOT including group name)
    timestamp: the timestamp (unix timestamp)
return token string for success, false for error


array fastdfs_get_file_info(string group_name, string filename)
get file info from the filename
parameters:
    group_name: the group name of the file
    remote_filename: the filename on the storage server
return assoc array for success, false for error. 
    the assoc array including following elements:
        create_timestamp: the file create timestamp (unix timestamp)
        file_size: the file size (bytes)
        source_ip_addr: the source storage server ip address


array fastdfs_get_file_info1(string file_id)
get file info from the file id
parameters:
    file_id: the file id (including group name and filename) or remote filename
return assoc array for success, false for error. 
    the assoc array including following elements:
        create_timestamp: the file create timestamp (unix timestamp)
        file_size: the file size (bytes)
        source_ip_addr: the source storage server ip address


bool fastdfs_send_data(int sock, string buff)
parameters:
    sock: the unix socket description
    buff: the buff to send
return true for success, false for error


string fastdfs_gen_slave_filename(string master_filename, string prefix_name
                [, string file_ext_name])
generate slave filename by master filename, prefix name and file extension name
parameters:
    master_filename: the master filename / file id to generate 
            the slave filename
    prefix_name: the prefix name  to generate the slave filename
    file_ext_name: slave file extension name, can be null or emtpy 
            (do not including dot)
return slave filename string for success, false for error


boolean fastdfs_storage_file_exist(string group_name, string remote_filename
    [, array tracker_server, array storage_server])
check file exist
parameters:
    group_name: the group name of the file
    remote_filename: the filename on the storage server
    tracker_server: the tracker server assoc array including elements: 
                        ip_addr, port and sock
    storage_server: the storage server assoc array including elements:
                        ip_addr, port and sock
return true for exist, false for not exist


boolean fastdfs_storage_file_exist1(string file_id
    [, array tracker_server, array storage_server])
parameters:
    file_id: the file id of the file
    tracker_server: the tracker server assoc array including elements: 
                        ip_addr, port and sock
    storage_server: the storage server assoc array including elements:
                        ip_addr, port and sock
return true for exist, false for not exist


array fastdfs_storage_upload_by_filename(string local_filename
    [, string file_ext_name, array meta_list, string group_name, 
    array tracker_server, array storage_server])
upload local file to storage server
parameters:
    local_filename: the local filename
    file_ext_name: the file extension name, do not include dot(.)
    meta_list: meta data assoc array, such as
                   array('width'=>1024, 'height'=>768)
    group_name: specify the group name to store the file
    tracker_server: the tracker server assoc array including elements: 
                        ip_addr, port and sock
    storage_server: the storage server assoc array including elements:
                        ip_addr, port and sock
return assoc array for success, false for error. 
       the returned array includes elements: group_name and filename


string fastdfs_storage_upload_by_filename1(string local_filename
    [, string file_ext_name, array meta_list, string group_name, 
    array tracker_server, array storage_server])
upload local file to storage server
parameters:
    local_filename: the local filename
    file_ext_name: the file extension name, do not include dot(.)
    meta_list: meta data assoc array, such as
                   array('width'=>1024, 'height'=>768)
    group_name: specify the group name to store the file
    tracker_server: the tracker server assoc array including elements: 
                        ip_addr, port and sock
    storage_server: the storage server assoc array including elements:
                        ip_addr, port and sock
return file_id for success, false for error.


array fastdfs_storage_upload_by_filebuff(string file_buff
    [, string file_ext_name, array meta_list, string group_name, 
    array tracker_server, array storage_server])
upload file buff to storage server
parameters:
    file_buff: the file content
    file_ext_name: the file extension name, do not include dot(.)
    meta_list: meta data assoc array, such as
                   array('width'=>1024, 'height'=>768)
    group_name: specify the group name to store the file
    tracker_server: the tracker server assoc array including elements: 
                        ip_addr, port and sock
    storage_server: the storage server assoc array including elements:
                        ip_addr, port and sock
return assoc array for success, false for error. 
       the returned array includes elements: group_name and filename


string fastdfs_storage_upload_by_filebuff1(string file_buff
    [, string file_ext_name, array meta_list, string group_name, 
    array tracker_server, array storage_server])
upload file buff to storage server
parameters:
    file_buff: the file content
    file_ext_name: the file extension name, do not include dot(.)
    meta_list: meta data assoc array, such as
                   array('width'=>1024, 'height'=>768)
    group_name: specify the group name to store the file
    tracker_server: the tracker server assoc array including elements: 
                        ip_addr, port and sock
    storage_server: the storage server assoc array including elements:
                        ip_addr, port and sock