7.18(RSYNC)

Linux RSYNC详解

一、linux系统下数据同步服务RSYNC
1、什么是rsync

rsync的好姐妹

■ sync同步:刷新文件系统缓存,强制将修改过的数据块写入磁盘,并且更新超级块。

■ async异步:将数据先放到缓冲区,再周期性(一般是30s)的去同步到磁盘。

■ rsync远程同步 :==remote synchronous==

数据同步过程

sync数据同步=>保存文件(目标)=>强制把缓存中的数据写入磁盘(立即保存),实
时性要求比较高的场景

asyn数据异步=>保存文件(目标)=>将数据先放到缓冲区,再周期性(一般是30s)的
去同步到磁盘,适合大批量数据同步的场景

2、rsync特点

■ 可以镜像保存整个目录树和文件系统|

■ 可以保留原有的权限(permission,mode),owner,group,时间(修改时间,modify time),
软硬链接,文件acl,文件属性(attributes)信息等

■ 传输 == 效率高 == ,使用同步算法,只比较变化的(增量备份)
file1.txt file2.txt file3.txt(A服务器)

rsync实现数据同步       =>只同步file3.txt=>增量备份

file1.txt file2.txt(B服务器)

■支持匿名传输,方便网站镜像;也可以做验证,加强安全

3、rsync与scp的区别

两者都可以实现远程同步,但是相对比而言,rsync能力更强

1.支持增量备份
2.数据同步时保持文件的原有属性

二、rsync的使用
1、安装RSYNC

[root@aa ~]# rpm -aq |grep rsync

[root@aa ~]#ls /etc/yum.repos.d/
[root@aa ~]# yum -y install rsync

2、基本语法
[root@aa ~]# which rsync
/usr/bin/rsync
[root@aa ~]# find / -name "*rsync"   查看安装位置

基本语法:rsync 【选项】 原数据位置   目录位置

选项    说明
-v    详细模式输出
-a    归档模式,递归的方式传输文件,并保持文件属性
-r    递归拷贝目录
-l    保留软链接
-p    保留原有地址
-t    保留原有时间(修改)
-g    保留属组权限
-o    保留属主权限
-D    等于--devices --specials 表示支持b,c,s,p类型文件
-R    保留相对路径
-H    保留硬链接
-A    保留ACL策略
-e    保留要执行的远程shell命令,ssh更改端口常用选项
-E    保留可执行权限
-X    保留扩展属性信息 a属性
PUSH 推,相当于上传; PULL 拉相当于下载

3、本地文件同步

3.1在家目录中创建文件,将文件同步到/opt下
[root@aa ~]# mkdir folder
[root@aa ~]# mkdir folder/f{1..3}
[root@aa ~]# tree folder/
folder/
├── f1
├── f2
└── f3
[root@aa ~]# touch folder/f1/file{0..4}
[root@aa ~]# tree folder/
folder/
├── f1
│   ├── file0
│   ├── file1
│   ├── file2
│   ├── file3
│   └── file4
├── f2
└── f3

3.2将folder目录下的文件传递/opt
[root@aa ~]# rsync -av folder/ /opt/
[root@aa ~]# ls /opt/

3.3将folder目录整体传递到/opt
[root@aa ~]# rsync -av folder /opt/   

3.4-R 保留相对路径
[root@aa ~]# rm -rf /opt/*
[root@aa ~]# rsync -avR folder/ /opt/

3.5现在不传递到opt,就在本地几个目录传

[root@aa ~]# tree folder/
[root@aa ~]# rsync -av folder/f1/ folder/f2/
[root@aa ~]# touch folder/f1/file5
[root@aa ~]# tree folder/
folder/
├── f1
│   ├── file0
│   ├── file1
│   ├── file2
│   ├── file3
│   ├── file4
│   └── file5
├── f2
│   ├── file0
│   ├── file1
│   ├── file2
│   ├── file3
│   └── file4
└── f3
[root@aa ~]# rsync -av folder/f1/ folder/f2/     //只同步file5
sending incremental file list
./
file5
[root@aa ~]# rm -rf folder/f1/file0      //删除file中file0
[root@aa ~]# tree folder/
[root@aa ~]# rsync -av folder/f1/ folder/f2/     //f2并没有同步
[root@aa ~]# rsync -av --delete folder/f1/ folder/f2/   //f2同步

4、修改文件

[root@aa ~]# vim folder/f1/file1    //修改文件内容
[root@aa ~]# cat folder/f1/file1
鲨鱼
[root@aa ~]# cat folder/f2/file1     
[root@aa ~]# rsync -av folder/f1/ folder/f2/   //文件的修改也会被rsync同步
[root@aa ~]# cat folder/f2/file1
鲨鱼
[root@aa ~]# touch folder/f1/file0 -m -d "2024-7-13 00:00"   //文件创建时间修改
[root@aa ~]# rsync -av folder/f1/ folder/f2/
[root@aa ~]# ls -l folder/f1/file0
-rw-r--r-- 1 root root 0 7月  13 00:00 folder/f1/file0

[root@aa ~]# ls -l folder/f2/file0      //f2会同步
-rw-r--r-- 1 root root 0 7月  13 00:00 folder/f2/file0
[root@aa ~]# chmod g+w folder/f1/file0    //修改文件所属组写权限
[root@aa ~]# ls -l folder/f1/file0
-rw-rw-r-- 1 root root 0 7月  13 00:00 folder/f1/file0
[root@aa ~]# rsync -av folder/f1/ folder/f2/
[root@aa ~]# ls -l folder/f2/file0      //f2会同步
-rw-rw-r-- 1 root root 0 7月  13 00:00 folder/f2/file0

5、向另一台主机 /tmp目录同步数据

远程同步:rsync -ac 源 用户@主机地址

获取数据pull拉取  push推送

[root@localhost ~]# yum -y install rsync    //另一台主机也需要安装rsync

[root@aa ~]# rsync -av folder/ root@192.168.8.132:/tmp/

[root@localhost ~]# ls /tmp/

[root@localhost ~]# dd if=/dev/zero of=/tmp/lajiwenjain bs=300M count=1
[root@localhost ~]# ls /tmp/ -lh

[root@aa ~]# rsync -av root@192.168.8.132:/tmp/lajiwenjain /tmp/   //从远程主机拉取数据

[root@aa ~]# ls -lh /tmp/ | grep lajiwenjain
-rw-r--r-- 1 root root 300M 7月  18 11:04 lajiwenjain

6、rsync作为系统服务
6.1查找rsync主配置文件


[root@aa ~]# systemctl start rsyncd
[root@aa ~]# netstat -lntup |grep rsync    //使用873端口

[root@aa ~]# find / -name "rsync*conf"   //找配置文件

/etc/rsyncd.conf

[root@aa ~]# vim /etc/rsyncd.conf

# uid = nobody    用户编号

# gid = nobody    所属组编号

# use chroot = yes    是否禁锢目录

# max connections = 4    最大连接数

# pid file = /var/run/rsyncd.pid    进程文件,每个服务都有一个PID编号,其来源就是PID文件

# exclude = lost+found/   传输文件时,忽略lost+fiund/目录

# transfer logging = yes    传输日志是否需要写入日志文件,yes表示写入

# timeout = 900     传输超时时间,默认15分钟

# ignore nonreadable = yes    是否忽略不可读文件

# dont compress   = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2   以下格式文件不压缩

[root@aa ~]# mkdir -p /app/studentweb/src/main/java/co/goho/yuanyu.studentweb/
[root@aa ~]# tree /app/
/app/
└── studentweb
    └── src
        └── main
            └── java
                └── co
                    └── goho
                        └── yuanyu.studentweb
[root@aa ~]# touch /app/studentweb/src/main/java/co/goho/yuanyu.studentweb/File{0..9}.java

[root@aa ~]# tree /app/


6.2把rsync作为系统服务运行


在aa主机中提供了一个针对app/下的项目rsync服务,下一次直接找服务 

[root@aa ~]# cd /app/studentweb/     //复制路径
[root@aa studentweb]# vim /etc/rsyncd.conf

[root@aa studentweb]# systemctl restart rsyncd

[root@localhost ~]# rsync -a root@192.168.8.131::
app 

[root@localhost ~]# rsync -a root@192.168.8.131::app /tmp/   拉取

[root@localhost ~]# ls -l /tmp/ |grep src
drwxr-xr-x. 3 root root        18 7月  18 11:28 src
[root@localhost ~]# tree /tmp/src/

7、自动化推送拉去文件


每30秒推送一次

编辑计划任务 crontab -e

[root@aa ~]# which rsync
/usr/bin/rsync
[root@aa ~]# crontab -e

[root@localhost ~]# rm -rf /tmp/*
[root@localhost ~]# ls /tmp/

自动监听文件被修改行为,在自动推送

三、RSYNC课程扩展
1、给rsyncd服务添加密码
1.1修改配置文件添加两个属性   

auth users=user0,user1    (账号和免密路径) 

secrets file=/etc/rsync.secrets

[root@aa ~]# vim /etc/rsyncd.conf

1.2编辑rsync密码
[root@aa ~]# vim /etc/resync.secrets

1.3给/etc/resync.secrets添加权限并重启

[root@aa ~]# chmod 600 /etc/resync.secrets

[root@aa ~]# ls -l /etc/resync.secrets
-rw------- 1 root root 28 7月  18 15:11 /etc/resync.secrets
[root@aa ~]# systemctl restart rsyncd

1.4拉取数据
[root@localhost ~]# rsync -av zhangsan@192.168.8.131::app /tmp/

2、rsync集合inotify工具实现代码实时同步
2.1直接安装inotify-tools
监听指定目录,一旦目录发生修改,就执行指令命令

[root@aa ~]# yum -y install inotify-tools

2.2inotify 监听
安装完后,就会产生下面两个命令
/usr/bin/inotifywait    等待
/usr/bin/inotifywatch   看守

[root@aa ~]# inotifywait -mr /app/     

[root@aa ~]# touch /app/studentweb/woshiqinshihuan.txt   

另开终端创建文件,inotifywhit监听到目录改变就会被推送

2.3编写inotify.sh
[root@aa ~]# vim inotifytest

选项    说明
-m    一直监控某个目录,create,delete,modify等行为
-r    递归,不仅仅监控目录还要监控目录下的文件
-q    获取操作信息,但是不能输出
-e    哪些行为需要被监控,modify,delete,create,attrib,move
modify    文件被修改
delete    文件被删除
create    文件被创建
attrib    文件属性被修改
move    文件被移动
[root@aa ~]# mv inotifytest inotifytest.sh
[root@aa ~]# chmod 700 inotifytest.sh 

[root@aa ~]# ./inotifytest.sh

另开一个终端创建文件,会监听信息

[root@aa ~]# touch /app/studentweb/nihaowoshiqingshihuang.txt

2.4让inotifytest.sh文件在后台一直运行
nohup:在后台一直运行,即使终端关闭

&:让inotify.sh在计算机后台运行,可以使用jobs命令查看,kill %编号结束,当我
们退出终端时,这个执行会自动结束

[root@aa ~]# nohup ./inotifytest.sh &

3.如何查看rsync.log⽇志⽂件
[root@aa ~]## cat /var/log/rsync.log
————————————————

                            版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
                        
原文链接:https://blog.youkuaiyun.com/m0_71589190/article/details/140530564

/** * implement a container like std::map */ #ifndef SJTU_MAP_HPP #define SJTU_MAP_HPP #include <cstdio> // only for std::less<T> #include <functional> #include <cstddef> #include "utility.hpp" #include "exceptions.hpp" namespace sjtu { template< class Key, class T, class Compare = std::less <Key> > class AAtree{ public: typedef pair<const Key, T> value_type; struct Node { Node *lson, *rson, *parent; int level; value_type data; Node(const value_type &d, int lv = 1, Node *p = nullptr, Node *l = nullptr, Node *r = nullptr) : data(d), level(lv), parent(p), lson(l), rson(r) {} }; Node *root; size_t tree_size; Compare comp; AAtree(): root(nullptr), tree_size(0) {} AAtree(const AAtree &other) { root = copyTree(other.root, nullptr); tree_size = other.tree_size; } AAtree &operator=(const AAtree &other) { if (this != &other) { clear(root); root = copyTree(other.root, nullptr); tree_size = other.tree_size; } return *this; } ~AAtree() { clear(root); tree_size = 0; } Node *skew(Node *node) { if (!node || !node->lson) return node; if (node->lson->level == node->level) { Node *L = node->lson; node->lson = L->rson; if (L->rson) L->rson->parent = node; L->rson = node; L->parent = node->parent; node->parent = L; return L; } return node; } Node *split(Node *node) { if (!node || !node->rson || !node->rson->rson) return node; if (node->level == node->rson->rson->level) { Node *R = node->rson; node->rson = R->lson; if (R->lson) R->lson->parent = node; R->lson = node; R->parent = node->parent; node->parent = R; ++R->level; return R; } return node; } Node *insert(Node *node, const value_type &value, Node *parent = nullptr) { if (!node) { ++tree_size; return new Node(value, 1, parent); } if (comp(value.first, node->data.first)) { node->lson = insert(node->lson, value, node); node->lson->parent = node; } else if (comp(node->data.first, value.first)) { node->rson = insert(node->rson, value, node); node->rson->parent = node; } else return node; node = skew(node); node = split(node); return node; } Node *erase(Node *node, const Key &key) { if (!node) return nullptr; if (comp(key, node->data.first)) { node->lson = erase(node->lson, key); if (node->lson) node->lson->parent = node; } else if (comp(node->data.first, key)) { node->rson = erase(node->rson, key); if (node->rson) node->rson->parent = node; } else { if (!node->lson && !node->rson) { if (node->parent) { if (node->parent->lson == node) { node->parent->lson = nullptr; } else { node->parent->rson = nullptr; } } delete node; node = nullptr; --tree_size; return nullptr; } else if (!node->lson) { Node *temp = node->rson; temp->parent = node->parent; delete node; node = nullptr; --tree_size; return temp; } else if (!node->rson) { Node *temp = node->lson; temp->parent = node->parent; delete node; node = nullptr; --tree_size; return temp; } Node *predecessor = node->lson; while (predecessor->rson) predecessor = predecessor->rson; Node *newNode = new Node(predecessor->data, node->level, node->parent, node->lson, node->rson); if (newNode->lson) newNode->lson->parent = newNode; if (newNode->rson) newNode->rson->parent = newNode; if (newNode->parent) { if (newNode->parent->lson == node) { newNode->parent->lson = newNode; } else { newNode->parent->rson = newNode; } } newNode->lson = erase(newNode->lson, predecessor->data.first); if (newNode->lson) newNode->lson->parent = newNode; delete node; node = newNode; } if (node->lson && node->rson) { size_t min_level = node->lson->level < node->rson->level ? node->lson->level : node->rson->level; if (node->level > min_level + 1) { node->level = min_level + 1; if (node->rson && node->rson->level > node->level) node->rson->level = node->level; } } node = skew(node); node->rson = skew(node->rson); if (node->rson) node->rson->rson = skew(node->rson->rson); node = split(node); node->rson = split(node->rson); return node; } Node *find(Node *node, const Key &key) const { while (node) { if (comp(key, node->data.first)) node = node->lson; else if (comp(node->data.first, key)) node = node->rson; else return node; } return nullptr; } Node *findMin(Node *node) const { if (!node) return nullptr; // printf("In findmin: %d\n", (node)); while (node->lson) node = node->lson; // printf("finish find min\n"); return node; } Node *findMax(Node *node) const { if (!node) return nullptr; while (node->rson) node = node->rson; return node; } Node *successor(Node *node) const { // printf("in successor\n"); if (!node) return nullptr; // printf("in successor2\n"); if (node->rson) return findMin(node->rson); // printf("in successor3\n"); Node *parent = node->parent; while (parent && node == parent->rson) { node = parent; parent = parent->parent; } // printf("finish successor\n"); return parent; } Node *predecessor(Node *node) const { if (!node) return nullptr; if (node->lson) return findMax(node->lson); Node *parent = node->parent; while (parent && node == parent->lson) { node = parent; parent = parent->parent; } return parent; } Node *copyTree(Node *node, Node *parent) { if (!node) return nullptr; Node *newNode = new Node(node->data, node->level, parent); newNode->lson = copyTree(node->lson, newNode); newNode->rson = copyTree(node->rson, newNode); return newNode; } void clear(Node *node) { if (!node) return; clear(node->lson); clear(node->rson); delete node; node = nullptr; } }; template< class Key, class T, class Compare = std::less <Key> > class map { public: /** * the internal type of data. * it should have a default constructor, a copy constructor. * You can use sjtu::map as value_type by typedef. */ typedef pair<const Key, T> value_type; AAtree<Key, T, Compare> aa_tree; /** * see BidirectionalIterator at CppReference for help. * * if there is anything wrong throw invalid_iterator. * like it = map.begin(); --it; * or it = map.end(); ++end(); */ typedef typename AAtree<Key, T, Compare>::Node Node; class const_iterator; class iterator { private: /** * TODO add data members * just add whatever you want. */ map* container; Node *node; public: iterator(map *c = nullptr, Node *n = nullptr): container(c), node(n) { // TODO } iterator(const iterator &other):container(other.container), node(other.node) { // TODO } /** * TODO iter++ */ iterator operator++(int) { if (!node) throw invalid_iterator(); iterator temp = *this; node = container->aa_tree.successor(node); return temp; } /** * TODO ++iter */ iterator &operator++() { if (!node) throw invalid_iterator(); node = container->aa_tree.successor(node); return *this; } /** * TODO iter-- */ iterator operator--(int) { iterator temp = *this; if (node == nullptr) { node = container->aa_tree.findMax(container->aa_tree.root); if (!node) throw invalid_iterator(); } else { node = container->aa_tree.predecessor(node); if (!node) throw invalid_iterator(); } return temp; } /** * TODO --iter */ iterator &operator--() { if (node == nullptr) { node = container->aa_tree.findMax(container->aa_tree.root); if (!node) throw invalid_iterator(); } else { node = container->aa_tree.predecessor(node); if (!node) throw invalid_iterator(); } return *this; } /** * a operator to check whether two iterators are same (pointing to the same memory). */ value_type &operator*() const { if (!node) throw invalid_iterator(); return node->data; } bool operator==(const iterator &rhs) const { return node == rhs.node && container == rhs.container; } bool operator==(const const_iterator &rhs) const { return node == rhs.getNode() && container == rhs.getContainer(); } /** * some other operator for iterator. */ bool operator!=(const iterator &rhs) const { return node != rhs.node || container != rhs.container; } bool operator!=(const const_iterator &rhs) const { return node != rhs.getNode() || container != rhs.getContainer(); } map *getContainer() const { return container; } Node *getNode() const { return node; } /** * for the support of it->first. * See <http://kelvinh.github.io/blog/2013/11/20/overloading-of-member-access-operator-dash-greater-than-symbol-in-cpp/> for help. */ value_type *operator->() const noexcept { // if (!node) throw invalid_iterator(); return &(node->data); } }; class const_iterator { // it should has similar member method as iterator. // and it should be able to construct from an iterator. private: // data members. map* container; Node *node; public: // const_iterator(const map *c = nullptr, Node *n = nullptr): container(const_cast<map*>(c)), node(n) { // // TODO // } const_iterator(map *c = nullptr, Node *n = nullptr): container(c), node(n) { // TODO } const_iterator(const const_iterator &other): container(other.container), node(other.node) { // TODO } const_iterator(const iterator &other) { // TODO container = other.getContainer(); node = other.getNode(); } /** * TODO iter++ */ const_iterator operator++(int) { if (!node) throw invalid_iterator(); const_iterator temp = *this; node = container->aa_tree.successor(node); return temp; } /** * TODO ++iter */ const_iterator &operator++() { if (!node) throw invalid_iterator(); node = container->aa_tree.successor(node); return *this; } /** * TODO iter-- */ const_iterator operator--(int) { const_iterator temp = *this; if (node == nullptr) { node = container->aa_tree.findMax(container->aa_tree.root); if (!node) throw invalid_iterator(); } else { node = container->aa_tree.predecessor(node); if (!node) throw invalid_iterator(); } return temp; } /** * TODO --iter */ const_iterator &operator--() { if (node == nullptr) { node = container->aa_tree.findMax(container->aa_tree.root); if (!node) throw invalid_iterator(); } else { node = container->aa_tree.predecessor(node); if (!node) throw invalid_iterator(); } return *this; } /** * a operator to check whether two iterators are same (pointing to the same memory). */ const value_type &operator*() const { if (!node) throw invalid_iterator(); return node->data; } bool operator==(const iterator &rhs) const { return node == rhs.getNode() && container == rhs.getContainer(); } bool operator==(const const_iterator &rhs) const { return node == rhs.node && container == rhs.container; } /** * some other operator for iterator. */ bool operator!=(const iterator &rhs) const { return node != rhs.getNode() || container != rhs.getContainer(); } bool operator!=(const const_iterator &rhs) const { return node != rhs.node || container != rhs.container; } map *getContainer() const { return container; } Node *getNode() const { return node; } /** * for the support of it->first. * See <http://kelvinh.github.io/blog/2013/11/20/overloading-of-member-access-operator-dash-greater-than-symbol-in-cpp/> for help. */ const value_type *operator->() const noexcept { // if (!node) throw invalid_iterator(); return &(node->data); } }; /** * TODO two constructors */ map(): aa_tree() {} map(const map &other): aa_tree(other.aa_tree) {} /** * TODO assignment operator */ map &operator=(const map &other) { if (this != &other) { aa_tree = other.aa_tree; } return *this; } /** * TODO Destructors */ ~map() {} /** * TODO * access specified element with bounds checking * Returns a reference to the mapped value of the element with key equivalent to key. * If no such element exists, an exception of type `index_out_of_bound' */ T &at(const Key &key) { auto node = aa_tree.find(aa_tree.root, key); if (!node) throw index_out_of_bound(); return node->data.second; } const T &at(const Key &key) const { auto node = aa_tree.find(aa_tree.root, key); if (!node) throw index_out_of_bound(); return node->data.second; } /** * TODO * access specified element * Returns a reference to the value that is mapped to a key equivalent to key, * performing an insertion if such key does not already exist. */ T &operator[](const Key &key) { auto node = aa_tree.find(aa_tree.root, key); if (node) return node->data.second; aa_tree.root = aa_tree.insert(aa_tree.root, value_type(key, T())); return aa_tree.find(aa_tree.root, key)->data.second; } /** * behave like at() throw index_out_of_bound if such key does not exist. */ const T &operator[](const Key &key) const { return at(key); } /** * return a iterator to the beginning */ iterator begin() { return iterator(this, aa_tree.findMin(aa_tree.root)); } const_iterator cbegin() const { return const_iterator(const_cast<map*>(this), aa_tree.findMin(aa_tree.root)); } /** * return a iterator to the end * in fact, it returns past-the-end. */ iterator end() { return iterator(this, nullptr); } const_iterator cend() const { return const_iterator(const_cast<map*>(this), nullptr); } /** * checks whether the container is empty * return true if empty, otherwise false. */ bool empty() const { return aa_tree.tree_size == 0; } /** * returns the number of elements. */ size_t size() const { return aa_tree.tree_size; } /** * clears the contents */ void clear() { aa_tree.clear(aa_tree.root); aa_tree.root = nullptr; aa_tree.tree_size = 0; } /** * insert an element. * return a pair, the first of the pair is * the iterator to the new element (or the element that prevented the insertion), * the second one is true if insert successfully, or false. */ pair<iterator, bool> insert(const value_type &value) { auto node = aa_tree.find(aa_tree.root, value.first); if (node) return {iterator(this, node), false}; aa_tree.root = aa_tree.insert(aa_tree.root, value); node = aa_tree.find(aa_tree.root, value.first); return {iterator(this, node), true}; } /** * erase the element at pos. * * throw if pos pointed to a bad element (pos == this->end() || pos points an element out of this) */ void erase(iterator pos) { if (pos == end() || pos.getContainer() != this) throw invalid_iterator(); aa_tree.root = aa_tree.erase(aa_tree.root, (*pos).first); } /** * Returns the number of elements with key * that compares equivalent to the specified argument, * which is either 1 or 0 * since this container does not allow duplicates. * The default method of check the equivalence is !(a < b || b > a) */ size_t count(const Key &key) const { return aa_tree.find(aa_tree.root, key) ? 1 : 0; } /** * Finds an element with key equivalent to key. * key value of the element to search for. * Iterator to an element with key equivalent to key. * If no such element is found, past-the-end (see end()) iterator is returned. */ iterator find(const Key &key) { return iterator(this, aa_tree.find(aa_tree.root, key)); } const_iterator find(const Key &key) const { return const_iterator(const_cast<map*>(this), aa_tree.find(aa_tree.root, key)); } }; } #endif 下面是valgrind的报错: ==9160== Invalid read of size 4 ==9160== at 0x10FDB9: check13() (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x110CC9: main (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== Address 0x6c8bcfc is 28 bytes inside a block of size 40 free'd ==9160== at 0x484BB6F: operator delete(void*, unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==9160== by 0x1150A1: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x115001: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x115001: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x115001: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x115292: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x115001: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x114F98: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x115001: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x115001: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x115001: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x115001: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== Block was alloc'd at ==9160== at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==9160== by 0x1147E3: sjtu::AAtree<int, int, std::less<int> >::insert(sjtu::AAtree<int, int, std::less<int> >::Node*, sjtu::pair<int const, int> const&, sjtu::AAtree<int, int, std::less<int> >::Node*) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x114850: sjtu::AAtree<int, int, std::less<int> >::insert(sjtu::AAtree<int, int, std::less<int> >::Node*, sjtu::pair<int const, int> const&, sjtu::AAtree<int, int, std::less<int> >::Node*) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x1148A7: sjtu::AAtree<int, int, std::less<int> >::insert(sjtu::AAtree<int, int, std::less<int> >::Node*, sjtu::pair<int const, int> const&, sjtu::AAtree<int, int, std::less<int> >::Node*) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x1148A7: sjtu::AAtree<int, int, std::less<int> >::insert(sjtu::AAtree<int, int, std::less<int> >::Node*, sjtu::pair<int const, int> const&, sjtu::AAtree<int, int, std::less<int> >::Node*) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x114850: sjtu::AAtree<int, int, std::less<int> >::insert(sjtu::AAtree<int, int, std::less<int> >::Node*, sjtu::pair<int const, int> const&, sjtu::AAtree<int, int, std::less<int> >::Node*) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x1148A7: sjtu::AAtree<int, int, std::less<int> >::insert(sjtu::AAtree<int, int, std::less<int> >::Node*, sjtu::pair<int const, int> const&, sjtu::AAtree<int, int, std::less<int> >::Node*) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x1148A7: sjtu::AAtree<int, int, std::less<int> >::insert(sjtu::AAtree<int, int, std::less<int> >::Node*, sjtu::pair<int const, int> const&, sjtu::AAtree<int, int, std::less<int> >::Node*) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x1148A7: sjtu::AAtree<int, int, std::less<int> >::insert(sjtu::AAtree<int, int, std::less<int> >::Node*, sjtu::pair<int const, int> const&, sjtu::AAtree<int, int, std::less<int> >::Node*) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x1148A7: sjtu::AAtree<int, int, std::less<int> >::insert(sjtu::AAtree<int, int, std::less<int> >::Node*, sjtu::pair<int const, int> const&, sjtu::AAtree<int, int, std::less<int> >::Node*) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x1148A7: sjtu::AAtree<int, int, std::less<int> >::insert(sjtu::AAtree<int, int, std::less<int> >::Node*, sjtu::pair<int const, int> const&, sjtu::AAtree<int, int, std::less<int> >::Node*) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x1148A7: sjtu::AAtree<int, int, std::less<int> >::insert(sjtu::AAtree<int, int, std::less<int> >::Node*, sjtu::pair<int const, int> const&, sjtu::AAtree<int, int, std::less<int> >::Node*) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== Test 13 Passed! Test 14 Passed! ==9160== ==9160== HEAP SUMMARY: ==9160== in use at exit: 0 bytes in 0 blocks ==9160== total heap usage: 325,886 allocs, 325,886 frees, 13,111,520 bytes allocated ==9160== ==9160== All heap blocks were freed -- no leaks are possible ==9160== ==9160== ERROR SUMMARY: 4308 errors from 1 contexts (suppressed: 0 from 0) ==9160== ==9160== 4308 errors in context 1 of 1: ==9160== Invalid read of size 4 ==9160== at 0x10FDB9: check13() (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x110CC9: main (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== Address 0x6c8bcfc is 28 bytes inside a block of size 40 free'd ==9160== at 0x484BB6F: operator delete(void*, unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==9160== by 0x1150A1: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x115001: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) ==9160== by 0x115001: sjtu::AAtree<int, int, std::less<int> >::erase(sjtu::AAtree<int, int, std::less<int> >::Node*, int const&) (in /root/code/STLite/STLite-ACM-2025/map/build/map_three_men) map_three_men: bool check13(){ // erase(it--) sjtu::map<int, int> Q; std::map<int, int> stdQ; int num[30001]; num[0] = 0; for(int i = 1; i <= 30000; i++) num[i] = num[i - 1] + rand() % 325 + 1; for(int i = 1; i <= 60000; i++) swap(num[rand() % 30000 + 1], num[rand() % 30000 + 1]); for(int i = 1; i <= 30000; i++){ int t = rand(); stdQ[num[i]] = t; Q[num[i]] = t; } // cout << "Stage 1" << endl; sjtu::map<int, int>::iterator it; std::map<int, int>::iterator stdit; for(int i = 1; i <= 60000; i++) swap(num[rand() % 30000 + 1], num[rand() % 30000 + 1]); for(int i = 1; i <= 10325; i++){ it = Q.find(num[i]); if(it != Q.begin()) { Q.erase(it--); // cout << "Erase it" << endl; } stdit = stdQ.find(num[i]); if(stdit != stdQ.begin()) { stdQ.erase(stdit--); // cout << "Erase it" << endl; } // cout << "i = " << i << " it, std = " << (it)->first << " " << stdit->first << endl; if(it -> first != stdit -> first)return 0; } // cout << "Stage 2" << endl; if(Q.size() != stdQ.size()) return 0; it = Q.begin(); // cout << "Stage 3" << endl; for(stdit = stdQ.begin(); stdit != stdQ.end(); stdit++){ if(stdit -> first != it -> first) return 0; if(stdit -> second != (*it).second) return 0; it++; } stdit = --stdQ.end(); for(it = --Q.end(); it != Q.begin(); it--){ if(stdit -> first != it -> first) return 0; if(stdit -> second != (*it).second) return 0; stdit--; } return 1; }
最新发布
05-28
<think>好的,我现在需要解决用户提到的C++中自定义AA树实现的map容器在erase操作时出现的valgrind内存错误问题,具体是Invalidread ofsize4。首先,我得回忆一下Valgrind的这个错误通常意味着什么。Invalidread通常表示程序试图访问已经释放的内存,或者越界访问。而size4可能是指读取了一个int类型的变量,因为int在32位系统通常是4字节。用户使用的是自己实现的AA树作为map容器,erase操作时出现问题。AA树是一种自平衡二叉搜索树,erase操作涉及节点删除和树的再平衡。内存错误可能出现在节点删除过程中没有正确管理指针或内存释放。 首先,可能的问题点有几个方面: 1. **悬空指针**:在删除节点后,可能还有指针指向已释放的内存。例如,父节点或兄弟节点仍然引用已删除的节点,导致后续操作读取了无效内存。 2. **重复释放**:同一块内存被释放多次,可能在删除节点时没有正确置空指针,导致后续再次释放。 3. **内存泄漏**:相反,如果节点没有被正确释放,也可能导致其他问题,但valgrind的Invalidread更可能是访问已释放内存。接下来,我需要考虑AA树erase的具体实现。通常,删除节点需要处理几种情况:节点是叶子节点、有一个子节点、或有两个子节点。在AA树中,删除可能涉及 skew和 split操作来维持平衡。可能的错误点: -**未正确更新父节点指针**:删除节点后,父节点的左右指针可能仍指向被删除的节点,导致后续访问时读取无效内存。- **未正确处理节点的子节点**:在删除有两个子节点的节点时,通常会找到后继节点替换,并删除后继节点。如果后继节点的处理不当,可能导致指针错误。- **未在删除后正确调整树结构**:AA树的平衡操作(skew/split)可能在删除后未正确执行,导致树的结构出现异常,进而引发内存访问错误。 需要检查代码中删除节点的逻辑,特别是释放内存后是否将所有指向该节点的指针置为nullptr,或者是否确保没有其他地方保留该节点的引用。例如,在删除节点时,如果父节点没有及时更新子指针,就可能出现悬空指针。 另外,参考用户提供的引用内容,他们之前可能遇到深拷贝问题。虽然当前问题与erase相关,但深拷贝问题可能导致对象复制时资源管理不当。但当前问题更可能出现在删除逻辑,而不是拷贝构造。不过,如果节点的数据成员在拷贝时没有正确管理,可能在删除时引发问题。例如,如果节点中的数据是浅拷贝的,删除时可能导致重复释放。根据用户提供的引用代码中的深拷贝示例,可能用户之前遇到过拷贝构造函数未正确实现的问题。如果在AA树的实现中,节点复制时没有正确进行深拷贝,可能导致多个节点指向同一块内存,删除其中一个节点后,其他节点仍引用该内存,导致Invalid read。不过,当前问题出现在erase操作,所以更应关注删除过程中的资源释放和指针管理。可能的解决方案步骤: 1. **检查删除节点的指针更新**:确保删除节点后,父节点、子节点的指针正确更新,避免留下悬空指针。2.**验证内存释放是否正确**:确保每个delete或delete[]操作对应正确的new/new[],并且没有重复释放。3.**使用Valgrind详细日志**:通过Valgrind的输出确定具体哪一行代码引发了Invalid read,从而定位到错误的源头。 4. **检查AA树平衡操作的实现**:skew和split操作可能在删除后未正确执行,导致树结构破坏,进而引发非法内存访问。例如,在删除节点后,若在平衡调整过程中访问了已经释放的节点,就会导致Invalidread。需要确保在调整树结构时,所有操作的节点都是有效的,未被释放的。另外,考虑使用智能指针管理节点内存,但可能在自定义数据结构中,手动管理更为常见,所以需要仔细检查手动管理的部分。可能的具体代码问题:假设节点结构类似:struct Node{Key key;Value value;Node*left;Node* right;int level; };在删除节点时,首先找到要删除的节点,处理其子节点,然后释放该节点内存。如果在释放后,某些操作(如平衡调整)仍然试图访问该节点,就会导致错误。 例如,在删除节点后,父节点的left或right指针未被及时置空,后续遍历时访问该指针就会读取已释放内存。 解决方案:在释放节点内存后,立即将父节点对应的指针置为nullptr,或者在删除过程中确保所有相关指针都正确更新。另一个可能的情况是,在删除节点时,未正确处理子节点的父指针。例如,如果节点有左子节点,在删除该节点后,左子节点的父指针需要正确指向新的父节点,否则可能在后续操作中访问到无效的父节点。 此外,在AA树的实现中,可能需要递归地删除节点,此时需要确保递归调用中没有使用已经释放的节点指针。 总结可能的解决步骤: 1.使用Valgrind定位具体出错的位置,查看调用堆栈,确定哪部分的代码引发了Invalidread。2.检查该处代码涉及的指针是否在删除后未被置空或未正确更新。 3.确保所有删除操作后,相关的父节点、子节点指针都被正确更新,避免遗留悬空指针。4.验证AA树的删除算法实现是否正确,特别是处理节点替换和平衡调整的部分。 例如,在删除含有两个子节点的节点时,通常需要找到后继节点(右子树的最小节点),然后用后继节点的值替换当前节点,再删除后继节点。如果在这个过程中,后继节点的父指针未正确更新,可能导致后续操作访问无效内存。另外,检查内存释放是否正确,例如对于Node结构体,是否使用delete而非delete[],反之如果节点内部分配了数组,是否使用delete[]。例如,如果节点中的key或value是动态分配的内存,并且在删除节点时没有正确释放这些内存,可能导致内存泄漏,但当前问题更可能是访问已释放的节点结构体本身。 最后,参考用户之前提到的深拷贝问题,如果在AA树的节点复制过程中(如果有的话)未正确实现深拷贝,可能导致多个节点共享同一资源,删除时重复释放。但通常map容器中的erase操作不涉及节点复制,而是直接删除,所以可能该问题与拷贝构造函数无关,除非在实现中有其他操作导致节点被错误复制。需要用户提供更多代码细节才能更准确判断,但基于现有信息,以上是可能的原因和解决方案方向。</think>### C++ AA树map容器erase操作内存错误解决方案 #### 问题定位与常见原因 Valgrind报告`Invalid read of size 4`通常由**访问已释放内存**或**指针未正确更新**引起。在AA树实现中,具体原因可能包括: 1. **悬空指针**:删除节点后,父节点或兄弟节点仍保留对已释放节点的引用[^1]。 2. **平衡操作错误**:`skew`或`split`操作中访问了已删除节点的子节点。 3. **后继节点处理不当**:替换被删除节点时,未正确更新父节点指针。 --- #### 分步解决方案 1. **验证指针更新逻辑** - 在释放节点前,确保父节点的`left`/`right`指针已置空: ```cpp void erase(Node* target) { Node* parent = find_parent(target); if (parent->left == target) { parent->left = nullptr; // 立即置空 } else { parent->right = nullptr; } delete target; // 最后释放内存 } ``` 2. **检查平衡操作** - 在`skew`和`split`函数中,添加空指针检查: ```cpp Node* skew(Node* node) { if (node == nullptr || node->left == nullptr) return node; // 平衡操作代码... } ``` 3. **后继节点替换的完整性** - 当用后继节点替换被删除节点时,需更新父节点指针: ```cpp Node* successor = find_min(node->right); node->key = successor->key; node->value = successor->value; erase(successor); // 递归删除需确保指针更新 ``` 4. **Valgrind调试辅助** - 通过`--track-origins=yes`定位未初始化值的来源: ```bash valgrind --tool=memcheck --leak-check=full --track-origins=yes ./your_program ``` --- #### 关键代码示例 ```cpp // 删除节点时正确处理父子关系 void erase_impl(Node*& node, const Key& key) { if (!node) return; if (key < node->key) { erase_impl(node->left, key); } else if (key > node->key) { erase_impl(node->right, key); } else { if (node->left && node->right) { // 处理有两个子节点的情况 Node* successor = find_min(node->right); node->key = successor->key; node->value = successor->value; erase_impl(node->right, successor->key); // 递归删除后继节点 } else { Node* temp = node; node = (node->left) ? node->left : node->right; delete temp; // 释放内存后,node指针已被上层更新 } } // 删除后重新平衡 if (node) { node = skew(node); node = split(node); } } ``` --- #### 验证方法 1. **单元测试**:覆盖删除根节点、叶子节点、内部节点的场景。 2. **Valgrind内存检查**:确保无`Invalid read`和`definitely lost`错误。 3. **压力测试**:插入/删除10万次后验证树的高度平衡性。 ---
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值