脚本实现全网数据备份搭建

最新推荐文章于 2025-06-13 12:59:49 发布

RSQ博客

最新推荐文章于 2025-06-13 12:59:49 发布

阅读量1.4k

点赞数 2

CC 4.0 BY-SA版权

分类专栏： Linux Shell 文章标签： NFS rsync inotify mail

本文链接：https://blog.youkuaiyun.com/Mr_rsq/article/details/79579247

Linux 同时被 2 个专栏收录

80 篇文章

订阅专栏

Shell

7 篇文章

订阅专栏

1 全网数据备份搭建整体思路

1.1 拓扑图

这里写图片描述

1.2 需求分析

1）3台服务器主机名分别为A(web01)、B(backup)、C(nfs01)

2）要求每晚00点整在Web服务器上打包备份系统配置文件、网站程序目录及访问日志并通过rsync命令推送备份服务器B上备份保留（备份思路可以是现在本地按日期打包，然后再推送到备份服务器B上）。

1.3 具体要求

Web01：

1）Web服务器A和备份服务器B的备份目录必须都为/backup

2）要求备份的系统配置文件包括但不限于：
a.定时任务服务的配置文件（/var/spool/cron/root）

b.开机自动启动的配置文件（/etc/rc.local）

c.日常脚本的目录（/server/scripts/）

d.防火墙iptables的配置文件（/etc/sysconfig/iptables）

e.Web服务器站点目录（/var/www/html）

f.Web服务器A访问日志路径假定为（/app/logs）

3）Web服务器保留打包后的7天的备份数据即可（本地保留不能多于7天，因为太多硬盘会满）

backup备份服务器：

1）备份服务器B上保留每周一的所有数据副本，其它要保留6个月的数据副本。

2）备份服务器B上要按照备份数据服务器的IP为目录保存备份，备份的文件按照时间名字保存。

3）要确保备份的数据尽量完整正确，在备份服务器上对备份的数据进行检查，把备份的成功及失败结果信息发送给系统管理员邮箱中。

NFS服务器：

1）在NFS服务端C(nfs01)上共享/data/w_shared及/data/r_shared两个文件目录，允许从NFS客户端A(web01)、B(backup)上分别挂载共享目录后可实现从A(web01)、B(backup)上只读/data/r_shared，可写/data/w_shared。

2）NFS客户端A(web01)上的挂载点为/data/b_w(写)，/data/b_r(读)

3）NFS客户端B(backup)上的挂载点为/data/w_你的名字英文(写)，/data/r_你的名字英文(读)。

4）从NFS客户端B(backup)上的NFS可写挂载点目录创建任意文件，从NFS客户端A(web01)上可以删除这个创建的文件，反之也可以。

5）优化NFS服务。

实时数据同步：

当用户通过web服务器将数据写入到NFS服务器C(nfs01)时，同时复制到备份服务器B(backup)

1.4 IP地址规划

服务器名称	内网IP	外网IP	主机名
Web服务	172.16.1.8/24	192.168.90.8/24	web01
NFS服务	172.16.1.31/24	192.168.90.31/24	nfs01
Backup备份服务	172.16.1.41/24	192.168.90.41/24	backup

2 实施步骤

2.1 系统优化脚本

#每个新的服务器上最好执行一遍，下面这个脚本只是简单的优化下系统

cat /server/scripts/youhua.sh
#!/bin/bash
#
export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Shudown iptables
/etc/init.d/iptables stop
chkconfig iptables off

#Shutdown SELinux
setenforce 0
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux.config
grep SELINUX=disabled /etc/selinux/config

#Simplify system services
export LANG=en
chkconfig |egrep -v "sshd|crond|network|rsyslog|sysstat" |awk '{print "chkconfig",$1,"off"}' |bash
chkconfig --list | grep "3:on"

#Sudo config
useradd oldboy
cp /etc/sudoers /etc/sudoers.ori
echo "oldboy ALL=(ALL) NOPASSWD:ALL " >>/etc/sudoers
tail -1 /etc/sudoers
visudo -c

#Time sync
echo '#time sync by oldboy' >>/var/spool/cron/root
echo "*/5 * * * * /usr/sbin/ntpdate ntp1.aliyun.com > /dev/null 2>&1" >>/var/spool/cron/root
crontab -l

#Command line setting
echo 'export TMOUT=300' >>/etc/profile
echo 'export HISTSIZE=5' >>/etc/profile
echo 'export HISTFILESIZE=5' >>/etc/profile
tail -3 /etc/profile
source /etc/profile

#Increase the description of the file
echo '*     -       nofile          65535' >> /etc/security/limits.conf
tail -1 /etc/security/limits.conf

#Kernel tuning
cat >>/etc/sysctl.conf<<EOF
net.ipv4.tcp_fin_timeout = 2
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_keepalive_time = 600
net.ipv4.ip_local_port_range = 4000 65000
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.route.gc_timeout = 100
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_synack_retries = 1
net.core.somaxconn = 16384
net.core.netdev_max_backlog = 16384
net.ipv4.tcp_max_orphans = 16384
net.nf_conntrack_max = 25000000
net.netfilter.nf_conntrack_max = 25000000
net.netfilter.nf_conntrack_tcp_timeout_established = 180
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
EOF
sysctl -p

#Update the yum source
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-6.repo
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo

2.2 主机名解析脚本

cat >>/etc/hosts<< EOF
172.16.1.5  lb01
172.16.1.6  lb02
172.16.1.7  web01
172.16.1.8  web02
172.16.1.51 db01 db01.etiantian.org
172.16.1.31 nfs01
172.16.1.41 backup
172.16.1.61 m01
EOF

2.3 backup服务器

2.3.1 搭建rsync服务

cat /server/scripts/rsyncser.sh
#!/bin/bash
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
user=rsync
authuser=rsync_backup
passwd=123456
passwdfile=/etc/rsync.password
bakpath1=/backup
bakpath2=/nfsbackup

#Install rsync
yum -y install rsync


#Create user
useradd $user -s /sbin/nologin -M


#Create configfile
cat >>/etc/rsyncd.conf<<EOF
#rsync_config_________________start
#created by rsq
##rsyncd.conf start##
uid = rsync
gid = rsync
use chroot = no
max connections = 200
timeout = 300
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
ignore errors
read only = false
list = false
hosts allow = 172.16.1.0/24
host deny = 0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/rsync.password
[backup]
path = /backup/

[nfsbackup]
path = /nfsbackup
#rsync_config_________________end
EOF


#Create passwdfile
echo "$authuser:$passwd" > $passwdfile
chmod 600 $passwdfile


#Create backup path
mkdir -p $bakpath1
mkdir -p $bakpath2
chown -R $user. $bakpath1
chown -R $user. $bakpath2


#Onboot
echo "/usr/bin/rsync --daemon" >>/etc/rc.local


#Start daemon
rsync --daemon
lsof -i :873

2.3.2 邮件搭建及检查脚本

#简单搭建邮件

cat >>/etc/mail.rc<<EOF
set from=cactirsq@163.com smtp=smtp.163.com smtp-auth-user=cactirsq smtp-auth-password=xxxxxxxx smtp-auth=login
EOF

#md5检查web01是否传送成功，没有丢包，有的话发邮件

cat >>/server/scripts/check.sh<<EOF
#!/bin/bash
#Check web01 backcup data
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
bakpath=/backup
find $bakpath -type f -name "flag_$(date +%F).log"|xargs md5sum -c $flagfile >>$bakpath/$(date +%F)_result.log 2>&1
mail -s "$(date +%F) backup result" 960503480@qq.com < $bakpath/$(date +%F)_result.log && \
find $bakpath -type f ! -name "*_1.tar.gz" -mtime +180|xargs rm -f
EOF

2.3.3 backup添加定时任务

cat >>/var/spool/cron/root<<EOF
#check for web01 backup data
00 04 * * * /bin/bash /server/scripts/check.sh &> /dev/null
EOF

2.4 web01服务器

2.4.1 web01 rsync客户端配置

cat /server/scripts/rsyncclient.sh
#!/bin/bash
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
authuer=rsync_backup
passwd=123456
passwdfile=/etc/rsync.password
bakpath=/backup

#Install rsync
yum -y install rsync


#Create passwdfile
echo "$passwd" >>$passwdfile
chmod 600 $passwdfile


#create backup path
mkdir -p $bakpath

2.4.2 web01创建所需的目录

#由于没有安装web服务，只是做简单的测试，故需要创建一开始不存在的目录

mkdir -p /app/logs

mkdir -p /var/www/html

2.4.3 web01备份到backup的脚本

cat /server/scripts/backup.sh
#!/bin/bash
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
IP=$(ifconfig eth1 | awk -F'[ :]+' 'NR==2{print $4}')
bakpath=/backup
user=rsync_backup
bakserver=172.16.1.41
module=backup
passwdfile=/etc/rsync.password

if [ ! -e $bakpath/$IP ];then
    mkdir -p $bakpath/$IP
fi

if [ $(date +%w) -eq 1 ];then
    time=$(date +%F_%w)
else
    time=$(date +%F)
fi

cd / && \
tar zchf $bakpath/$IP/web_$time.tar.gz var/spool/cron/root etc/rc.local var/www/html app/logs etc/sysconfig/iptables server/scripts etc/sysctl.conf && \

md5sum $bakpath/$IP/web_$time.tar.gz > $bakpath/$IP/flag_$(date +%F).log &&\

rsync -az $bakpath $user@$bakserver::$module/ --password-file=$passwdfile

find $bakpath -type f -mtime +7 \( -name "*.tar.gz" -o -name "*.log" \)|xargs rm -f

2.4.4 为web服务添加定时任务

cat >>/var/spool/cron/root<<EOF
#backup 
00 00 * * * /bin/bash /server/scripts/backup.sh
EOF

2.5 NFS服务器

2.5.1 搭建NFS服务

cat /server/scripts/nfsser.sh 
#!/bin/bash
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
share1=/data/r_shared
share2=/data/w_shared
user=nfsnobody
group=nfsnobody
IP=$(ifconfig eth1 | awk -F'[ :]+' 'NR==2{print $4}')
net=$(ifconfig eth1 | awk -F'[ :]+' 'NR==2{print $4}'|cut -d"." -f1-3)
mask=".0/24"
opt1="ro,sync,all_squash,anonuid=65534,anongid=65534"
opt2="rw,sync,all_squash,anonuid=65534,anongid=65534"


#Install nfs
yum -y install rpcbind nfs-utils


#Share folder
mkdir -p $share1 $share2
chown -R $user.$group $share1
chown -R $user.$group $share2


#Kernel youhua
cat >>/etc/sysctl.conf<<EOF
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 1677216
net.core.wmem_max = 1677216
EOF
sysctl -p


#NFS config file
cat >>/etc/exports<<EOF
$share2 $net$mask($opt2)
$share1 $net$mask($opt1)
EOF


#start service and onboot
/etc/init.d/rpcbind start && \
/etc/init.d/nfs start
echo "/etc/init.d/rpcbind start" >>/etc/rc.local
echo "/etc/init.d/nfs start" >>/etc/rc.local


#Test
showmount -e $IP

2.5.2 NFS客户端搭建(web01,nfs01)

cat /server/scripts/nfsmount.sh 
#!/bin/bash
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
nfsserver=172.16.1.31
user=nfsnobody
group=nfsnobody
mountpoint1=/data/b_r
mountpoint2=/data/b_w
mountopt="nosuid,noexec,nodev,noatime,rsize=131072,wsize=131072"


#Install nfs
yum install -y rpcbind nfs-utils


#Only start rpcbind service
/etc/init.d/rpcbind start


#mkdir 
mkdir -p $mountpoint1 $mountpoint2
chown -R $user.$group $mountpoint1
chown -R $user.$group $mountpoint2


#mount 
share1=$(showmount -e $nfsserver|awk 'NR==2{print $1}')
share2=$(showmount -e $nfsserver|awk 'NR==3{print $1}')

mount -t nfs -o $mountopt $nfsserver:$share1 $mountpoint1
mount -t nfs -o $mountopt $nfsserver:$share2 $mountpoint2


#mount onboot
echo "mount -t nfs -o $mountopt $nfsserver:$share1 $mountpoint1" >>/etc/rc.local
echo "mount -t nfs -o $mountopt $nfsserver:$share2 $mountpoint2" >>/etc/rc.local

2.5.3 NFS实时同步

#安装inotify

yum -y install inotify-tools
rpm -qa | grep inotify-tools

#youhua
echo 655350 >/proc/sys/fs/inotify/max_user_watches
echo 655350 >/proc/sys/fs/inotify/max_queued_events

#Create passwd_file
echo 123456 > /etc/rsync.password

inotify.sh

#!/bin/bash
PATH=/data
IP=172.16.1.41

/usr/bin/inotifywait -mrq --format '%w%f' -e close_write,delete $PATH \
|while read file
do
    if [ -f $file ];then
      /usr/bin/rsync -az $file --delete rsync_backup@$IP::nfsbackup --password-file=/etc/rsync.password
    else
      cd $PATH && \
      /usr/bin/rsync -az ./ --delete rsync_backup@$IP::nfsbackup --password-file=/etc/rsync.password
    fi
done

#设置inotify服务脚本，即在/etc/init.d/目录下创建一个syncd文件

cat /etc/init.d/syncd
#!/bin/bash
#chkconfig: 2345 38 46
#############################################################################
#this scripts is inotify start or stop
#############################################################################
. /etc/rc.d/init.d/functions

function usage(){
    echo "$0 {start|stop}"
}

if [ $# -ne 1 ];then
    usage
    exit 1
fi

case "$1" in
  start)
    /bin/bash /server/scripts/inotify.sh &
    echo $$ >/var/run/inotify.pid
    if [ `ps -ef|grep inotify|wc -l` -gt 2 ];then
        action "inotify service is started" /bin/true
    else
        action "inotify service is started" /bin/false
        fi
        ;;

  stop)
    kill -9 `cat /var/run/inotify.pid` >/dev/null 2>&1
    pkill inotifywait
    sleep 2
    if [ `ps -ef|grep inotify|grep -v grep|wc -l` -eq 0 ];then
        action "inotify service is stoped" /bin/true
    else
        action "inotify service is stoped" /bin/false
    fi
    ;;

  *)
    usage
    exit 1

esac

#启动服务脚本

chmod +x /etc/init.d/syncd
chkconfig --add syncd
chkconfig syncd on
chkconfig --list syncd

# start
/etc/init.d/syncd start

3 故障排错

主要是脚本内容的原因，有IP地址写错、变量名字后边引用错误等。

客户端挂载nfs反了。