记一次从RDS恢复备份数据库到自建服务器-xtrabackup

MySQL运维实战之备份和恢复(8.5)xtrabackup恢复增量备份 作者:俊达恢复增量备份时,需要先对基础全量备份进行恢复,然后再依次按增量备份的时间进行恢复。 阅读详情

1、内网下载数据全备文件
备份的压缩包名为:irisdb.tar.gz

wget -c 'http://rdsbak-hz-v3.oss-cn-xxxxxxxxxxxxy8%3D' -O /app/tar-gz/irisdb.tar.gz

2、解压数据备份文件压缩包:
解压文件存放路径为: /app/irisdb-back

tar -xvf  /app/tar-gz/irisdb.tar.gz  -C  /app/irisdb-back >> /dev/null

确认成功解压正常完成。

3、进入数据库:新建数据库,初始化root用户密码

mysql> set password for 'root'@'localhost' = password('root');
Query OK, 0 rows affected (0.00 sec)

4、停数据库

service mysql3307 stop

5、挪走新安装好的MySQL原来的datadir下的数据,新建文件夹,并授权:操作对象./db目录

[root@hz-beta-mysql mysql]# mv /app/irisdb/mysql/db /app/irisdb/mysql/db-old-data
[root@hz-beta-mysql mysql]# mkdir -p /app/irisdb/mysql/db
[root@hz-beta-mysql mysql]# chown -R mysql:mysql /app/irisdb/mysql/db

6、恢复数据到数据目录(datadir=*,由my3307.cnf文件指定了 )(放后台运行)
先应用日志(apply-log),然后copy-back
使用innobackupex恢复数据,备份的数据目录为/app/irisdb-back/,
数据库的datadir 等根据参数 --defaults-file 指定的配置文件 为:/etc/my3307.cnf

/usr/local/bin/innobackupex --defaults-file=/etc/my3307.cnf --user=mysql --password=mysql --apply-log /app/irisdb-back/ >  recover-log.sql
/usr/local/bin/innobackupex --defaults-file=/etc/my3307.cnf --user=mysql --password=mysql --copy-back /app/irisdb-back/ >> recover-log.sql

apply-log 完成提示如下:

……
InnoDB: Setting log file ./ib_logfile1 size to 256 MB
InnoDB: Progress in MB: 100 200
InnoDB: Setting log file ./ib_logfile2 size to 256 MB
InnoDB: Progress in MB: 100 200
InnoDB: Setting log file ./ib_logfile3 size to 256 MB
InnoDB: Progress in MB: 100 200
InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
InnoDB: New log files created, LSN=79481523649052
InnoDB: Highest supported file format is Barracuda.
InnoDB: 128 rollback segment(s) are active.
InnoDB: Waiting for purge to start
InnoDB: 5.6.24 started; log sequence number 79481523649548
xtrabackup: starting shutdown with innodb_fast_shutdown = 1
InnoDB: FTS optimize thread exiting.
InnoDB: Starting shutdown...
InnoDB: Shutdown completed; log sequence number 79481523650973
190203 05:12:50 completed OK!

copy-back恢复完成会提示如下:

……
190131 06:44:19 [01] Copying ./sms_db/db.opt to /app/irisdb/mysql//db/sms_db/db.opt
190131 06:44:19 [01]        ...done
190131 06:44:19 [01] Copying ./sms_db/t_sms.frm to /app/irisdb/mysql//db/sms_db/t_sms.frm
190131 06:44:19 [01]        ...done
190131 06:44:19 [01] Copying ./sms_db/t_sms.ibd to /app/irisdb/mysql//db/sms_db/t_sms.ibd
190131 06:54:28 [01]        ...done
190131 06:54:28 [01] Copying ./_witness_t_fdd_witness_send_log_backup_main_41008_2_1b.tokudb to /app/basic/mysql//db/_witness_t_fdd_witness_send_log_backup_main_41008_2_1b.tokudb
190131 06:55:52 [01]        ...done
190131 06:55:52 completed OK!

注意修改恢复后的文件夹的权限,如果全县有问题,则会报错,如下

……
2019-01-31 09:31:44 29014 [Note] InnoDB: Using Linux native AIO
2019-01-31 09:31:44 29014 [Note] InnoDB: Using CPU crc32 instructions
2019-01-31 09:31:44 29014 [Note] InnoDB: Initializing buffer pool, size = 3.1G
2019-01-31 09:31:45 29014 [Note] InnoDB: Completed initialization of buffer pool
2019-01-31 09:31:45 29014 [ERROR] InnoDB: ./ibdata1 can't be opened in read-write mode
2019-01-31 09:31:45 29014 [ERROR] InnoDB: The system tablespace must be writable!
2019-01-31 09:31:45 29014 [ERROR] Plugin 'InnoDB' init function returned error.
2019-01-31 09:31:45 29014 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2019-01-31 09:31:45 29014 [ERROR] Unknown/unsupported storage engine: InnoDB
2019-01-31 09:31:45 29014 [ERROR] Aborting

7、编辑MySQL的配置文件
跳过密码验证

vim /etc/my3308.cnf
[mysqld]
skip-grant-tables

8、数据库服务启动

service mysql3307 start 

9、进入数据库修改用户密码

update user set password=password('root') where user='root';
flush privileges;

或
当库中 无root@localhost用户的时候

insert into user (user,host,password,ssl_type,ssl_cipher,x509_issuer,x509_subject) values ('root','localhost',password('root'),'','','','');
update user set
  Select_priv='Y'
, Insert_priv='Y'
, Update_priv='Y'
, Delete_priv='Y'
, Create_priv='Y'
, Drop_priv='Y'
, Reload_priv='Y'
, Shutdown_priv='Y'
, Process_priv='Y'
, File_priv='Y'
, Grant_priv='Y'
, References_priv='Y'
, Index_priv='Y'
, Alter_priv='Y'
, Show_db_priv='Y'
, Super_priv='Y'
, Create_tmp_table_priv='Y'
, Lock_tables_priv='Y'
, Execute_priv='Y'
, Repl_slave_priv='Y'
, Repl_client_priv='Y'
, Create_view_priv='Y'
, Show_view_priv='Y'
, Create_routine_priv='Y'
, Alter_routine_priv='Y'
, Create_user_priv='Y'
, Event_priv='Y'
, Trigger_priv='Y'
, Create_tablespace_priv='Y'
where user='root' and host='localhost';
flush privileges;

这种insert方式,所有权限都已经授权,注意操作user表处理权限时,要flush privileges 一下。

10、编辑配置文件,注释掉免密码登录,并重启数据库服务,使用用户名&密码登录

vim /etc/my3308.cnf
[mysqld]
# skip-grant-tables

重启数据库服务前,注意修改数据库目录权限

chown -R mysql:mysql ./irisdb/
service mysql3307 restart

否则报错如下:

……
2019-02-12 09:33:01 27205 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
2019-02-12 09:33:01 27205 [Note] InnoDB: Using atomics to ref count buffer pool pages
2019-02-12 09:33:01 27205 [Note] InnoDB: The InnoDB memory heap is disabled
2019-02-12 09:33:01 27205 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-02-12 09:33:01 27205 [Note] InnoDB: Compressed tables use zlib 1.2.3
2019-02-12 09:33:01 27205 [Note] InnoDB: Using Linux native AIO
2019-02-12 09:33:01 27205 [Note] InnoDB: Using CPU crc32 instructions
2019-02-12 09:33:01 27205 [Note] InnoDB: Initializing buffer pool, size = 3.5G
2019-02-12 09:33:02 27205 [Note] InnoDB: Completed initialization of buffer pool
2019-02-12 09:33:02 27205 [ERROR] InnoDB: ./ibdata1 can't be opened in read-write mode
2019-02-12 09:33:02 27205 [ERROR] InnoDB: The system tablespace must be writable!
2019-02-12 09:33:02 27205 [ERROR] Plugin 'InnoDB' init function returned error.
2019-02-12 09:33:02 27205 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2019-02-12 09:33:02 27205 [ERROR] Unknown/unsupported storage engine: InnoDB
2019-02-12 09:33:02 27205 [ERROR] Aborting

11、登录数据库,验证

mysql -uroot -p --default-character-set=utf8 --socket=/app/irisdb/mysql/mysql.sock  

12、新建用户

grant select ,update ,delete,insert on iris.* to 'user_seiki'@'%' identified by 'test_190307';

遇到的问题及解决方案:

报错如下:

[root@hz-beta-mysql bin]#
/app/xtra-install/percona-xtrabackup-2.3.2-Linux-x86_64/bin/xtrabackup
–defaults-file=/app/irisdb-back/backup-my.cnf --user=root --password=root --apply-log /app/irisdb/mysql/db/ Warning: Using unique option prefix apply-log instead of apply-log-only is deprecated
and will be removed in a future release. Please use the full name
instead. xtrabackup: Error: unknown argument: ‘/app/irisdb/mysql/db/’

–target-dir:指的是目标目录,即计划备份数据是,数据存储的目标目录位置
/app/xtra-install/percona-xtrabackup-2.3.2-Linux-x86_64/bin/xtrabackup --defaults-file=/app/irisdb-back/backup-my.cnf --user=root --password=root /app/irisdb/mysql-backup/

写错了会报错

/app/xtra-install/percona-xtrabackup-2.3.2-Linux-x86_64/bin/xtrabackup version 2.3.2 based on MySQL server 5.6.24 Linux (x86_64) (revision id: 306a2e0)
2019-01-25 16:53:50 7f65dd198720  InnoDB: Operating system error number 2 in a file operation.
InnoDB: The error means the system cannot find the path specified.
[01] error: cannot open file ibdata1
2019-01-25 16:53:50 7f65dd198720  InnoDB: Operating system error number 9 in a file operation.
InnoDB: Error number 9 means 'Bad file descriptor'.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/operating-system-error-codes.html
InnoDB: File (unknown): 'close' returned OS error 109. Cannot continue operation
2019-01-25 16:53:50 7f65dd198720  InnoDB: Assertion failure in thread 140075477862176 in file os0file.cc line 658
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
08:53:50 UTC - xtrabackup got signal 6 ;
This could be because you hit a bug or data is corrupted.
This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
记一次阿里云RDS MYSQL 数据恢复的经历 方法有两种! 情况1: 如果阿里云RDS备份时间在你删除数据之后,那么根据日志备份里的binlog文件里来查找记录,那个是全的. 1.1 下载日志文件中的bin文件 1.2 在linux下 用下面的代码解码出来 xxxx 是日志的数字,不要在windows下!!! win下出来的中文是乱码的,网上你都找不到这个问题,都是提示你加base64. mysqlbinlog -vv --bas... 阅读详情

相关推荐

阿里云rds物理备份文件恢复到自建数据库

阿里云rds物理备份文件恢复到自建数据库 开源软件Percona Xtrabackup可以用于对数据库进行备份恢复; 前提条件: 数据库:MySQL 5.6 系统:Linux7.6的操作系统 开源软件Percona Xtrabackup可以用于对数据库进行备份恢复,操作系统中已安装数据恢复工具Percona XtraBackup。 MySQL 5.6及之前的版本需要安装 Percon...

mwtai的博客 1014

xtrabackup: error: last checkpoint LSN (3409281307) is larger than last copied LSN (3409274368). #2

1.错误发生场景:使用2.4.1版本的xtrabackup工具进行全备,备份日志中报出此错误2.知识要点:MySQL中,redo 日志写进程会在三种条件下被触发从log buffer中写日志到redo log file中,分别是每隔一秒、日志达到log buffer的1/3、每一次提交的时候。刷新日志的LSN到log file是采用轮询的方式进行的,比如有三组日志当这三组都被写满的时候再...

T99887766的博客 526

腾迅云MySQL RDS备份恢复到本地数据库测试

MySQL版本:5.6 步骤1:下载备份文件 RDS管理控制台提供下载链接,如果库不太大,可以直接下载。 如果库比较大,建议使用同一区域服务器,从内网下在到服务器,再切割打包为较小文件下载。 步骤2:解包备份文件 xbstream -x -C /dbdata02< db02.xb /dbdata02是数据解包后存放的路径 db02.xb是从腾迅云下载的数据库备份文件 如图中,解包文件中以“.qp”结尾的文件是经过压缩处理的数据文件。要使用前还需要进行解压缩处理。 步骤3:解压缩.

bigwood99的博客 672

记录一次阿里云RDS-Mysql备份文件,本地windows数据库恢复过程

恢复云数据库MySQL的备份文件到自建数据库 https://help.aliyun.com/knowledge_detail/41817.html 官方给的是linux系统的恢复教程现在我们本地mysql数据库是windows系统的mysql,以下是下载rds备份包在本地的恢复过程1. 下载解压备份包2. 发现数据库文件夹中都是opt,frm,ibd后缀的文件3. 将mysql系统无关数据库文件...

hbh112233abc的博客 4131

QGIS中高效处理CAD数据的两种实用技巧

本文详细解析了在QGIS中高效处理CAD数据的两种核心方法。针对DWG/DXF文件在QGIS中直接打开失败的问题,介绍了使用QGIS内置数据导入管理器进行标准化转换的流程,并重点推荐了功能更强大的Another DXF Importer插件,以更好地保留文字、块属性等复杂信息。文章对比了两种方法的优劣与适用场景,并提供了坐标系设置、中文乱码等常见问题的避坑指南,帮助用户根据数据需求选择最佳导入策略。

weixin_28363123的博客 117

RK3588 EVB开发板原理图讲解【六】

该原理图展示了eMMC Flash与主控芯片之间的连接关系,以及相关的电源、信号线路和去耦电容等。

weixin_48721556的博客 1169

xtrabackup备份失败问题解决

前言 下面总结的是部分常见的报错及解决方法。 常见报错原因 【1】使用的xtrabackup安装失败或软件版本与系统不适配。 【2】备份主库时,从库上有业务操作,且出现超大事务。 【3】备份使用的账号的权限不足,可使用管理员权限。 【4】mysql的参数配置问题,例如:连接超时时间配置较长等。 【5】备份时使用的参数配置错误异常导致。 【6】系统中的程序或脚本问题,kill掉了mysq...

漠效的博客 4688

告别迷茫!手把手教你用Synopsys VC USB VIP搭建验证平台(附UVM分层序列实战解析)

本文详细介绍了如何使用Synopsys VC USB VIP搭建验证平台,并深入解析UVM分层序列的实战应用。通过Step-by-Step的操作指南和关键配置示例,帮助验证工程师高效集成USB VIP,掌握从环境搭建到测试用例开发的全套技能,特别适合处理复杂协议验证的挑战。

weixin_42520573的博客 553

xtrabackup

xtrabackup 使用总结:1、会使用默认的参数文件2、xtrabackup --copy-back   --target-dir=/mysql_back      进行restore 时,有些文件不会迁移,例如 xtrabackup_checkpoints,下面是我做了copy-back 后再做 prepare,报错了[root@localhost mysql_data]# xtraback...

huangliang0703的专栏 2481

python操作git推送自己分支并合并主分支(持续优化该工具)

python git

qq_44707265的博客 862

LeGo-LOAM源码篇:源码分析(3)

mapOptmization.cpp,主要内容包括回环检测、scan-to-map位姿优化、因子图。

slamml 1382

RDS MySQL和Mongodb 物理备份文件.xb恢复到自建数据库

一:准备环境 1:已经有了RDS的.xb文件,如未有,请参考此连接 2:已经搭建好自建数据库,如果未有,请参考此连接其中mysql搭建方式 Centos7安装percona-xtrabackup2.4和8.0版本 a cat /mnt/software/practice_qp_2021.xb | xbstream -x -v -C /mnt/lzx/mysql/data .........

李指导的博客 6980

2025虚幻游戏逆向工程解包尝试

2025虚幻游戏逆向工程解包指南 本文详细介绍了虚幻引擎5游戏逆向解包的完整流程,主要包括获取AES密钥、生成映射文件和导出资源三个关键步骤。作者在实际操作中遇到诸多难题:AES密钥获取工具存在加壳保护问题,映射文件生成器依赖复杂的VS编译环境,DLL注入工具的使用也存在技术门槛。通过多次尝试,作者发现部分游戏可直接搜索现成的AES密钥,而映射文件生成需要额外下载Memcury依赖库才能正常编译。文章提供了相关工具的下载链接和具体操作说明,但同时也指出每个环节都可能出现意外状况,逆向工程实际执行过程远比理论

wsdchong的博客 6150

阿里云RDS数据恢复到本地MySQL数据库

参考自阿里云 恢复云数据库MySQL的备份文件到自建数据库, 本文补全了工具下载及安装过程,以及踩过的一些坑 1.工具下载 RDS MySQL 使用开源软件 percona-Xtrabackup 对MySQL 数据库进行备份。需要下载该软件,然后使用该软件进行恢复。下载percona-toolkit https://www.percona.com/downloads/percona-toolkit

刘老高的专栏 3923

(四) OpenClaw-Skill技能全面了解和自定义技能

OpenClaw Skills是提升AI代理功能性的关键模块。本文系统介绍了Skills的核心概念、获取渠道和创建方法。Skills作为标准化指令集合,能让OpenClaw从单纯理解升级为实际执行任务,相比传统System Prompt具有模块化、可复用等优势。用户可通过Clawhub官方商店、GitHub社区等渠道获取5700+现成技能,或按需开发自定义Skill。文章详细演示了创建"Hello World"示例技能的完整流程,包括编写Python脚本、配置SKILL.md文件、部署。

直到世界的尽头 8295

pytorch实现基本的logistic和softmax回归实验(手动+torch)

前提:不是前馈神经网络,没有隐藏层。 一:Logistic回归实验–人工构造数据集,手动构造模型 要求: 动手从0实现logistic回归,实现二分类,人工构造数据集,并分析loss、训练集、测试集的准确率。(要求从零实现二元交叉熵) 实验过程: 1.1人工构造数据集 1. # #自定义数据---训练集 2. num_inputs = 2 3. n_data = torch.ones(1500, num_inputs) # 数据的基本形态 4. x1 = torch.normal(2 * n_d

qq_37534947的博客 1万+

RDS数据库备份恢复

https://www.cnblogs.com/zoulixiang/p/9395382.html博客园https://help.aliyun.com/knowledge_detail/41817.html官方文章 ### 阿里云RDS数据库备份恢复到自建数据库 - 简单的方法一个是通过阿里云备份恢复到某一时间节点 但是需要新建一个RDS实例 需要花钱 - 本文讲的是下载备份恢复到本...

weixin_41053978的博客 169

pscad IEEE标准系统搭建说明

pscad IEEE标准系统搭建说明,包含典型的IEEE 9节点、14节点、30节点、39节点和118节点系统。内含标准测试系统的原始数据和潮流结果。

阿里云rds恢复到自建数据库

1.先去阿里元RDS 下载数据库备份文件 官方文档:https://help.aliyun.com/knowledge_detail/41817.html(或多或少总是有坑) 安装mysql 线下环境(centos7+mysql5.7) 注意如果是centos7 的话,安装mysql需要先卸载mariadb,否则会有冲突 2.安装开源备份工具 yum install percona-xtrabac...

long的博客 307

visio安装包

Microsoft Office Visio 是一款专业的制图软件。利用经过更新的形状、协作工具和数据链接图表,创建专业级图表以简化复杂的信息。

IMSL CNL 7.0.0 (x86-64)Crack

IMSL CNL 7.0.0 (x86-64)Crack intel的算法库破解文件,这是C++版本的破解文件

阿里云RDS云数据库的物理备份文件恢复到本地Mysql数据库

虽然阿里云RDS数据库有定时全量备份的功能,但我还是在每个星期去页面上下载一份备份文件到本地服务器上,尽量做到异地灾备。。 领导检查数据的备份情况时,希望我把备份下来的数据恢复到本地数据库。果然是实时看得到才安心啊,哈哈。 其实这还是比较简单的,阿里云的帮助文档上也有,但还是在这里记录和分享下吧。 一、环境准备 1.安装本mysql 本地mysql要与RDS数据库所用得版本一致,且本地mysq...

嘻哈王大头 2501

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

RDS恢复其中某个数据库备份的流程

1.先开通了一台linux系统云服务器,通过宝塔面板安装好mysql8.0,将下载的rds物理备份auto_269_xxx.tar.gz上传到服务器。1.首先登录管理中心》业务管理》云数据库MYSQL》点管理进去,点备份,选择需要的备份文件,下载到本地或者一台云服务器上(以下教程为云服务器)。5.通过Navicat 导入恢复,参考:https://www.west.cn/faq/list.asp?4.导出单个库,参考:https://www.west.cn/faq/list.asp?

wwwwestcn的博客 314

恢复阿里云数据库MySQL的备份文件到自建数据库

恢复阿里云数据库MySQL的备份文件到自建数据库云数据库MySQL版使用开源软件Percona Xtrabackup对数据库进行备份,所以可以使用该软件将云数据库MySQL的备份文件恢复到自建数据库中一、前提条件1、本地MySQL数据库安装在64位的Linux系统中,且与云数据库MySQL版的版本相同(mysql安装文档这里就不叙述了)2、操作系统中已安装数据恢复工具Percona XtraBac...

神圣坤的博客 3018

如何将阿里云mysql RDS备份文件恢复到自建数据库

参考地址:https://help.aliyun.com/knowledge_detail/41817.html PS:目前恢复只支持 Linux 下进行。Linux下恢复的数据文件,无论 Windows、Linux 操作系统,MySQL 都可以正常使用 1、概述和软件说明 1.1 概述   RDS MySQL 使用开源软件 percona-Xtrabackup 对 MySQL ...

231

rds mysql 恢复到本地_阿里云RDS数据库备份文件恢复到本地mysql数据库

一、安装mysql和xtrabackup(1)安装mysql因为RDS是5.6版本,所以我们本地的mysql数据库要与RDS版本对应。rpm -ivh http://repo.mysql.com/mysql-community-release-el6-5.noarch.rpm && yum -y install mysql-server mysql(2)安装xtrabackup看我...

weixin_36079903的博客 589

阿里云数据库RDS MySQL 物理全备文件数据恢复至自建数据库Mysql 5.7中

环境介绍: CentOS 7.5 Mysql 5.7 percona-xtrabackup-24 注意事项 操作系统中已安装数据恢复工具Percona XtraBackup,您可以从Percona XtraBackup官网下载安装。 MySQL 5.6及之前的版本需要安装 Percona XtraBackup 2.3,安装指导请参见官方文档Percona XtraBackup 2.3。 MySQL 5.7版本需要安装 Percona XtraBackup 2.4,安装指导请参见官方文档Percona Xt

qq_40907977的博客 998

企业实战(15)阿里云数据库RDS MySQL 物理全备文件数据恢复至自建数据库Mysql 5.7中

环境介绍: CentOS 7.5 Mysql 5.7 注意事项  操作系统中已安装数据恢复工具Percona XtraBackup,您可以从Percona XtraBackup官网下载安装。 MySQL 5.6及之前的版本需要安装 Percona XtraBackup 2.3,安装指导请参见官方文档Percona XtraBackup 2.3。 MySQL 5.7版本需要安装 Percona XtraBackup 2.4,安装指导请参见官方文档Percona XtraBackup 2.4。 MySQL 8

非著名运维的博客 976

[mysql]-xtrabackup恢复阿里云RDS

上个周末上新项目 迁移数据库30个G的库 迁移了2次耗时1天.感觉效率太低,想用xtrabackup恢复,发现之前mysql5.6和mysql8.0的文章写得潦草.自己一时半会也有点看不懂了.于是重写一遍5.7的 这边使用内存1G 差不多1个小时就恢复30G的数据,使用dump恢复需要至少4小时半天 首先安装Percona XtraBackup 安装yum 源 Mysql5...

爷来辣的博客 758
上一篇: windows环境MySQL8.0的data目录变更方法及问题
下一篇: Mac 下 vmware fusion 安装配置 centos7
世樹
博客等级 码龄15年 37粉丝 45原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值