mydumper是一个针对MySQL的高性能多线程备份和恢复工具,它提供了并发备份功能,备份效率有很大提高。

安装mydumper

  • yum安装

    # rpm -ivh https://github.com/maxbube/mydumper/releases/download/v0.9.5/mydumper-0.9.5-2.el7.x86_64.rpm
  • 编译安装
    # yum install glib2-devel mysql-devel zlib-devel pcre-devel openssl-devel
    # wget https://github.com/maxbube/mydumper/archive/v0.9.5.tar.gz
    # tar xvf v0.9.5.tar.gz
    # cd mydumper-0.9.
    # cmake .
    # make

备份

mydumper备份时不是像mysqldump一样指定到某个特定的sql文件,而需要指定一个备份目录:

mydumper -h ${host} --user ${user} --password ${passwd}  -B ${DB_NAME} -o ${bakdir} -e -t 16

mydumper --help
Usage:
mydumper [OPTION...] multi-threaded MySQL dumping
Help Options:
-?, --help Show help options
Application Options:
-B, --database Database to dump
# 导出指定库
-T, --tables-list Comma delimited table list to dump (does not exclude regex option)
# 导出指定的表,多个表之间用逗号分隔。db1.t1,db1,t2,db2,t1,db3,t1
-O, --omit-from-file File containing a list of database.table entries to skip, one per line (skips before applying regex option)
-o, --outputdir Directory to output files to
# 指定备份目录
-s, --statement-size Attempted size of INSERT statement in bytes, default 1000000
# 指定导出SQL文件中每个insert语句的大小,默认每个insert 1000000value
-r, --rows Try to split tables into chunks of this many rows. This option turns off --chunk-filesize
-F, --chunk-filesize Split tables into chunks of this output file size. This value is in MB
-c, --compress Compress output files
# 压缩备份,导出SQL文件以sql.gz结尾
-e, --build-empty-files Build dump files even if no data available from table
# 如果空表也生成对应的文件名
-x, --regex Regular expression for 'db.table' matching
正则匹配库和表
-i, --ignore-engines Comma delimited list of storage engines to ignore
# 忽略不需要导出指定存储引擎的表
-N, --insert-ignore Dump rows with INSERT IGNORE
# 导出的SQL文件加上INSERT IGNORE
-m, --no-schemas Do not dump table schemas with the data
# 不导表结构,只导数据
-d, --no-data Do not dump table data
# 不导数据,只导表结构
-G, --triggers Dump triggers
# 备份触发器
-E, --events Dump events
# 备份事件
-R, --routines Dump stored procedures and functions
# 备份存储过程和函数
-W, --no-views Do not dump VIEWs
# 不导视图
-k, --no-locks Do not execute the temporary shared read lock. WARNING: This will cause inconsistent backups
# 备份时不加全局读锁,
--no-backup-locks Do not use Percona backup locks
# 不使用 Percona backup locks
--less-locking Minimize locking time on InnoDB tables.
-l, --long-query-guard Set long query timer in seconds, default 60
# 设置执行多长时间SQL将会被KILL
-K, --kill-long-queries Kill long running queries (instead of aborting)
# kill长时间执行的SQL
# -D, --daemon Enable daemon mode
以守护进程进行备份,默认每60分钟备份一次
-I, --snapshot-interval Interval between each dump snapshot (in minutes), requires --daemon, default 60
# 每个备份之间的时间间隔(以分钟为单位)需要--daemon,默认值为60
-L, --logfile Log file name to use, by default stdout is used
# 指定备份过程中的日志记录,默认标准输出
--tz-utc SET TIME_ZONE='+00:00' at top of dump to allow dumping of TIMESTAMP data when a server has data in different time zones or data is being moved between servers with different time zones, defaults to on use --skip-tz-utc to disable.
# 备份时时区设置,默认在备份文件开始添加SET TIME_ZONE='+00:00'语句
--skip-tz-utc
# 禁止在备份文件中输出SET TIME_ZONE语句
--use-savepoints Use savepoints to reduce metadata locking issues, needs SUPER privilege
# 使用savepoints来减少元数据锁定问题,需要SUPER权限
--success-on-1146 Not increment error count and Warning instead of Critical in case of table doesn't exist
--lock-all-tables Use LOCK TABLE for all, instead of FTWRL
-U, --updated-since Use Update_time to dump only tables updated in the last U days
--trx-consistency-only Transactional consistency only
--complete-insert Use complete INSERT statements that include column names
# 导出的备份文件带完整的字段名
-h, --host The host to connect to
-u, --user Username with the necessary privileges
-p, --password User password
-a, --ask-password Prompt For User password
# 交互输入密码
-P, --port TCP/IP port to connect to
-S, --socket UNIX domain socket file to use for connection
-t, --threads Number of threads to use, default 4
-C, --compress-protocol Use compression on the MySQL connection
# 备份传输使用压缩
-V, --version Show the program version and exit
-v, --verbose Verbosity of output, 0 = silent, 1 = errors, 2 = warnings, 3 = info, default 2
--defaults-file Use a specific defaults file
# 指定mysql my.cnf配置文件

备份完成后,可以查看备份目录下metadata文件,查看备份是否成功。

# cat metadata
Started dump at: -- ::
SHOW MASTER STATUS:
Log: mysql-bin.
Pos:
GTID:xxxxxxxx SHOW SLAVE STATUS:
Host: 192.168.1.10
Log: mysql-bin.
Pos:
GTID:xxxxx Finished dump at: -- ::

恢复

在恢复之前首先把sql_mode设置为空,否则会报错

mysql> show global variables like '%sql_mode%';
+---------------+------------------------------------------------------------------------------------------------------------------------+
| Variable_name | Value |
+---------------+------------------------------------------------------------------------------------------------------------------------+
| sql_mode | STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+---------------+------------------------------------------------------------------------------------------------------------------------+
mysql> set global sql_mode='';
执行完myloader之后再恢复设置
mysql> set global sql_mode='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';

myloader -u ${user} -p ${passwd} -h ${host} -o -t 8 -d  ${bakdir}  

myloader参数介绍:
-d, --directory # 导入备份目录
-q, --queries-per-transaction #每次执行的查询数量, 默认1000
-o, --overwrite-tables #如果表存在删除表
-B, --database # 需要还原的库
-s, --source-db Database to restore # 从备份文件把-s 指定的库恢复到数据库中
-e, --enable-binlog # 启用二进制恢复数据。恢复时是否开启binlog,默认导入时关闭binlog(有主从环境注意如果-e 那么从库也会一起恢复,但是速度较慢
-h, --host The host to connect to -u, --user Username with the necessary privileges
-p, --password User password
-a, --ask-password Prompt For User password
-P, --port TCP/IP port to connect to
-S, --socket UNIX domain socket file to use for connection #指定socket file
-t, --threads Number of threads to use, default 4 # 指定线程数,默认为4
-C, --compress-protocol Use compression on the MySQL connection
-V, --version Show the program version and exit
-v, --verbose Verbosity of output, 0 = silent, 1 = errors, 2 = warnings, 3 = info, default 2
--defaults-file Use a specific defaults file

备份恢复示例

master: 172.16.2.11    slave: 172.16.2.22

将master数据库的数据备份出来,恢复到slave里面,并配置主从同步。

1.在master创建备份用户和权限。
mysql> GRANT SELECT, RELOAD, PROCESS, SHOW DATABASES, SUPER, LOCK TABLES, REPLICATION SLAVE, REPLICATION CLIENT, SHOW VIEW, EVENT, TRIGGER ON *.* TO 'backup'@'172.16.2.22' identified by 'xxxxxx';
mysql > flush privileges;

2. 在slave备份远程库到本地
# mkdir /data/mydumper
# mydumper -G -E -R -h 172.16.2.11 --user backup --password xxxxx -o /data/mydumper/ -e -t 8

3.恢复。将mydumper备份的数据恢复到数据库里。
mysql> show global variables like '%sql_mode%';
+---------------+------------------------------------------------------------------------------------------------------------------------+
| Variable_name | Value |
+---------------+------------------------------------------------------------------------------------------------------------------------+
| sql_mode | STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+---------------+------------------------------------------------------------------------------------------------------------------------+
mysql> set global sql_mode='';
恢复步骤:
myloader -u root -p xxxxxx -o -t 16 -d /data/mydumper/ -S /tmp/mysql.sock

恢复完成后,将sql_mode还原:
mysql> set global sql_mode='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';

4.待数据恢复完成后,将slave和master配置成主从,同步数据。
mysql> CHANGE MASTER TO MASTER_HOST='172.16.2.11',MASTER_USER='repl',MASTER_PASSWORD='xxxxxx',MASTER_AUTO_POSITION=0,MASTER_LOG_FILE='mysql-bin.002714',MASTER_LOG_POS=73794931;
# binlog file和position在mydumper目录的metadata文件里找。
mysql> start slave;

最新文章

  1. echarts-案例
  2. iOS之小门道道
  3. 微信小程序-视图数据绑定
  4. C#获取CSV文件内容对逗号和引号分隔的处理
  5. RSuite 一个基于 React.js 的 Web 组件库
  6. 转:CDC,CPaintDC,CClientDC,CWindowDC区别
  7. PostgreSQL停止动作观察
  8. PLSQL Developer Debug
  9. 问题: Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable 解决方案
  10. Android - 分享内容
  11. Dockerfile详解
  12. 【机器学习实战】第 10 章 K-Means(K-均值)聚类算法
  13. 常见的磁盘I/O和网络I/O优化技巧
  14. 原生js添加博客点击鼠标出小心心效果~~
  15. Ubuntu 16.04 启用 点击Launcher图标,窗口实现最小化 功能
  16. jmeter实践之数据库参数传递
  17. sql server连接oracle并实现增删改查
  18. [SDOI2009]HH去散步 「矩阵乘法计数」
  19. 【Python】python3实现网页爬虫下载图片
  20. python文件的只读,只写操作

热门文章

  1. jar包中存在包名和类名都相同的情况
  2. BZOJ4543 Hotel加强版(长链剖分)
  3. BZOJ 2699: 更新 (DP)
  4. Luogu 4751 动态DP 模板
  5. perfectpixel 加载PSD图到网页中和已经写好的网页进行对比
  6. OFDM为什么把高频子载波作为保护频带
  7. springboot启动时执行任务CommandLineRunner
  8. Selenium+Python附件上传
  9. selenium鼠标操作
  10. 将页面中所有的checkbox设成单选得