近期zabbix数据库占用的io高,在页面查看图形很慢,而且数据表已经很大,将采用把数据库的数据目录移到新的磁盘,将几个大表进行分表操作

一、数据迁移:

1、数据同步到新的磁盘上,先停止mysql(不停止同步的话就有问题):
systemctl stop mariadb
rsync -av /var/lib/mysql/ /mysql_data/ 2、修改mysql的配置文件/etc/my.cnf:
datadir=/mysql_data 3、启动mysql:
systemctl stop mariadb

二、数据库分表:

1、查看表空间占用情况:
select table_name, (data_length+index_length)/1024/1024 as total_mb, table_rows from information_schema.tables where table_schema='zabbix'; 2、一般都是这几个表很大history、history_str、history_text、history_uint、trends、trends_uint,先为每个表创建空表(数据很大的话执行的时间太长):
history:
CREATE TABLE `history_20190619` (
`itemid` bigint(20) unsigned NOT NULL,
`clock` int(11) NOT NULL DEFAULT '',
`value` double(16,4) NOT NULL DEFAULT '0.0000',
`ns` int(11) NOT NULL DEFAULT '',
KEY `history_1` (`itemid`,`clock`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; history_str:
CREATE TABLE `history_str_20190619` (
`itemid` bigint(20) unsigned NOT NULL,
`clock` int(11) NOT NULL DEFAULT '',
`value` varchar(255) COLLATE utf8_bin NOT NULL DEFAULT '',
`ns` int(11) NOT NULL DEFAULT '',
KEY `history_str_1` (`itemid`,`clock`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; history_text:
CREATE TABLE `history_text_20190619` (
`itemid` bigint(20) unsigned NOT NULL,
`clock` int(11) NOT NULL DEFAULT '',
`value` text COLLATE utf8_bin NOT NULL,
`ns` int(11) NOT NULL DEFAULT '',
KEY `history_text_1` (`itemid`,`clock`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; history_uint:
CREATE TABLE `history_uint_20190619` (
`itemid` bigint(20) unsigned NOT NULL,
`clock` int(11) NOT NULL DEFAULT '',
`value` bigint(20) unsigned NOT NULL DEFAULT '',
`ns` int(11) NOT NULL DEFAULT '',
KEY `history_uint_1` (`itemid`,`clock`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; trends:
CREATE TABLE `trends_20190619` (
`itemid` bigint(20) unsigned NOT NULL,
`clock` int(11) NOT NULL DEFAULT '',
`num` int(11) NOT NULL DEFAULT '',
`value_min` double(16,4) NOT NULL DEFAULT '0.0000',
`value_avg` double(16,4) NOT NULL DEFAULT '0.0000',
`value_max` double(16,4) NOT NULL DEFAULT '0.0000',
PRIMARY KEY (`itemid`,`clock`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; trends_uint:
CREATE TABLE `trends_uint_20190619` (
`itemid` bigint(20) unsigned NOT NULL,
`clock` int(11) NOT NULL DEFAULT '',
`num` int(11) NOT NULL DEFAULT '',
`value_min` bigint(20) unsigned NOT NULL DEFAULT '',
`value_avg` bigint(20) unsigned NOT NULL DEFAULT '',
`value_max` bigint(20) unsigned NOT NULL DEFAULT '',
PRIMARY KEY (`itemid`,`clock`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin; 3、重命名表:
rename table history to history_back;
rename table history_20190619 to history; rename table history_str to history_str_back;
rename table history_str_20190619 to history_str; rename table history_text to history_text_back;
rename table history_text_20190619 to history_text; rename table history_uint to history_uint_back;
rename table history_uint_20190619 to history_uint; rename table trends to trends_back;
rename table trends_20190619 to trends; rename table trends_uint to trends_uint_back;
rename table trends_uint_20190619 to trends_uint; 4、将下面sql复制到partition.sql中,执行:
mysql -uzabbix -pzabbix zabbix < partition.sql
5、添加到定时任务:
01 01 * * * mysql -uzabbix -pzabbix zabbix -e"CALL partition_maintenance_all('zabbix')" &>/dev/null
6、手动执行:
mysql -uzabbix -pzabbix zabbix -e "CALL partition_maintenance_all('zabbix')" &> /root/partition.log& innodb_file_per_table

分表sql

DELIMITER $$
CREATE PROCEDURE `partition_create`(SCHEMANAME varchar(64), TABLENAME varchar(64), PARTITIONNAME varchar(64), CLOCK int)
BEGIN
/*
SCHEMANAME = The DB schema in which to make changes
TABLENAME = The table with partitions to potentially delete
PARTITIONNAME = The name of the partition to create
*/
/*
Verify that the partition does not already exist
*/ DECLARE RETROWS INT;
SELECT COUNT(1) INTO RETROWS
FROM information_schema.partitions
WHERE table_schema = SCHEMANAME AND table_name = TABLENAME AND partition_description >= CLOCK; IF RETROWS = 0 THEN
/*
1. Print a message indicating that a partition was created.
2. Create the SQL to create the partition.
3. Execute the SQL from #2.
*/
SELECT CONCAT( "partition_create(", SCHEMANAME, ",", TABLENAME, ",", PARTITIONNAME, ",", CLOCK, ")" ) AS msg;
SET @sql = CONCAT( 'ALTER TABLE ', SCHEMANAME, '.', TABLENAME, ' ADD PARTITION (PARTITION ', PARTITIONNAME, ' VALUES LESS THAN (', CLOCK, '));' );
PREPARE STMT FROM @sql;
EXECUTE STMT;
DEALLOCATE PREPARE STMT;
END IF;
END$$
DELIMITER ;
DELIMITER $$
CREATE PROCEDURE `partition_drop`(SCHEMANAME VARCHAR(64), TABLENAME VARCHAR(64), DELETE_BELOW_PARTITION_DATE BIGINT)
BEGIN
/*
SCHEMANAME = The DB schema in which to make changes
TABLENAME = The table with partitions to potentially delete
DELETE_BELOW_PARTITION_DATE = Delete any partitions with names that are dates older than this one (yyyy-mm-dd)
*/
DECLARE done INT DEFAULT FALSE;
DECLARE drop_part_name VARCHAR(16); /*
Get a list of all the partitions that are older than the date
in DELETE_BELOW_PARTITION_DATE. All partitions are prefixed with
a "p", so use SUBSTRING TO get rid of that character.
*/
DECLARE myCursor CURSOR FOR
SELECT partition_name
FROM information_schema.partitions
WHERE table_schema = SCHEMANAME AND table_name = TABLENAME AND CAST(SUBSTRING(partition_name FROM 2) AS UNSIGNED) < DELETE_BELOW_PARTITION_DATE;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE; /*
Create the basics for when we need to drop the partition. Also, create
@drop_partitions to hold a comma-delimited list of all partitions that
should be deleted.
*/
SET @alter_header = CONCAT("ALTER TABLE ", SCHEMANAME, ".", TABLENAME, " DROP PARTITION ");
SET @drop_partitions = ""; /*
Start looping through all the partitions that are too old.
*/
OPEN myCursor;
read_loop: LOOP
FETCH myCursor INTO drop_part_name;
IF done THEN
LEAVE read_loop;
END IF;
SET @drop_partitions = IF(@drop_partitions = "", drop_part_name, CONCAT(@drop_partitions, ",", drop_part_name));
END LOOP;
IF @drop_partitions != "" THEN
/*
1. Build the SQL to drop all the necessary partitions.
2. Run the SQL to drop the partitions.
3. Print out the table partitions that were deleted.
*/
SET @full_sql = CONCAT(@alter_header, @drop_partitions, ";");
PREPARE STMT FROM @full_sql;
EXECUTE STMT;
DEALLOCATE PREPARE STMT; SELECT CONCAT(SCHEMANAME, ".", TABLENAME) AS `table`, @drop_partitions AS `partitions_deleted`;
ELSE
/*
No partitions are being deleted, so print out "N/A" (Not applicable) to indicate
that no changes were made.
*/
SELECT CONCAT(SCHEMANAME, ".", TABLENAME) AS `table`, "N/A" AS `partitions_deleted`;
END IF;
END$$
DELIMITER ;
DELIMITER $$
CREATE PROCEDURE `partition_maintenance`(SCHEMA_NAME VARCHAR(32), TABLE_NAME VARCHAR(32), KEEP_DATA_DAYS INT, HOURLY_INTERVAL INT, CREATE_NEXT_INTERVALS INT)
BEGIN
DECLARE OLDER_THAN_PARTITION_DATE VARCHAR(16);
DECLARE PARTITION_NAME VARCHAR(16);
DECLARE OLD_PARTITION_NAME VARCHAR(16);
DECLARE LESS_THAN_TIMESTAMP INT;
DECLARE CUR_TIME INT; CALL partition_verify(SCHEMA_NAME, TABLE_NAME, HOURLY_INTERVAL);
SET CUR_TIME = UNIX_TIMESTAMP(DATE_FORMAT(NOW(), '%Y-%m-%d 00:00:00')); SET @__interval = 1;
create_loop: LOOP
IF @__interval > CREATE_NEXT_INTERVALS THEN
LEAVE create_loop;
END IF; SET LESS_THAN_TIMESTAMP = CUR_TIME + (HOURLY_INTERVAL * @__interval * 3600);
SET PARTITION_NAME = FROM_UNIXTIME(CUR_TIME + HOURLY_INTERVAL * (@__interval - 1) * 3600, 'p%Y%m%d%H00');
IF(PARTITION_NAME != OLD_PARTITION_NAME) THEN
CALL partition_create(SCHEMA_NAME, TABLE_NAME, PARTITION_NAME, LESS_THAN_TIMESTAMP);
END IF;
SET @__interval=@__interval+1;
SET OLD_PARTITION_NAME = PARTITION_NAME;
END LOOP; SET OLDER_THAN_PARTITION_DATE=DATE_FORMAT(DATE_SUB(NOW(), INTERVAL KEEP_DATA_DAYS DAY), '%Y%m%d0000');
CALL partition_drop(SCHEMA_NAME, TABLE_NAME, OLDER_THAN_PARTITION_DATE); END$$
DELIMITER ;
DELIMITER $$
CREATE PROCEDURE `partition_verify`(SCHEMANAME VARCHAR(64), TABLENAME VARCHAR(64), HOURLYINTERVAL INT(11))
BEGIN
DECLARE PARTITION_NAME VARCHAR(16);
DECLARE RETROWS INT(11);
DECLARE FUTURE_TIMESTAMP TIMESTAMP; /*
* Check if any partitions exist for the given SCHEMANAME.TABLENAME.
*/
SELECT COUNT(1) INTO RETROWS
FROM information_schema.partitions
WHERE table_schema = SCHEMANAME AND table_name = TABLENAME AND partition_name IS NULL; /*
* If partitions do not exist, go ahead and partition the table
*/
IF RETROWS = 1 THEN
/*
* Take the current date at 00:00:00 and add HOURLYINTERVAL to it. This is the timestamp below which we will store values.
* We begin partitioning based on the beginning of a day. This is because we don't want to generate a random partition
* that won't necessarily fall in line with the desired partition naming (ie: if the hour interval is 24 hours, we could
* end up creating a partition now named "p201403270600" when all other partitions will be like "p201403280000").
*/
SET FUTURE_TIMESTAMP = TIMESTAMPADD(HOUR, HOURLYINTERVAL, CONCAT(CURDATE(), " ", '00:00:00'));
SET PARTITION_NAME = DATE_FORMAT(CURDATE(), 'p%Y%m%d%H00'); -- Create the partitioning query
SET @__PARTITION_SQL = CONCAT("ALTER TABLE ", SCHEMANAME, ".", TABLENAME, " PARTITION BY RANGE(`clock`)");
SET @__PARTITION_SQL = CONCAT(@__PARTITION_SQL, "(PARTITION ", PARTITION_NAME, " VALUES LESS THAN (", UNIX_TIMESTAMP(FUTURE_TIMESTAMP), "));"); -- Run the partitioning query
PREPARE STMT FROM @__PARTITION_SQL;
EXECUTE STMT;
DEALLOCATE PREPARE STMT;
END IF;
END$$
DELIMITER ; DELIMITER $$
CREATE PROCEDURE`partition_maintenance_all`(SCHEMA_NAME VARCHAR(32))
BEGIN
CALL partition_maintenance(SCHEMA_NAME, 'history', 90, 24, 14);
CALL partition_maintenance(SCHEMA_NAME, 'history_log', 90, 24, 14);
CALL partition_maintenance(SCHEMA_NAME, 'history_str', 90, 24, 14);
CALL partition_maintenance(SCHEMA_NAME, 'history_text', 90, 24, 14);
CALL partition_maintenance(SCHEMA_NAME, 'history_uint', 90, 24, 14);
CALL partition_maintenance(SCHEMA_NAME, 'trends', 730, 24, 14);
CALL partition_maintenance(SCHEMA_NAME, 'trends_uint', 730, 24, 14);
END$$
DELIMITER ;

最新文章

  1. Linux上成功编译CoreCLR源代码
  2. Nginx+CI出现404错误
  3. mysql之对视图的操作
  4. PHP5中常用的魔术函数有哪些,举例说明各自的用法。
  5. trap命令使用
  6. 自定义使用AVCaptureSession 拍照,摄像,载图
  7. google搜索新姿势
  8. 把自己的程序打成jar包,让别人调用
  9. php 查找数组中是否存在某项,并返回指定的字符串,可用于检查复选,单选等
  10. PDO(20161107)
  11. (二)图数据neo4j基本认识
  12. hdu 2973&quot;YAPTCHA&quot;(威尔逊定理)
  13. python爬虫点触验证码的识别思路(图片版)
  14. SOA总结(脑图图片)
  15. Linux kernel Programming - Advanced Char Driver Operations
  16. tensorRT 使用tensorflow的pb问价构建推理
  17. linux nfs服务配置挂载以及oracle使用nfs存储挂载注意事项
  18. 爬虫系列5:scrapy动态页面爬取的另一种思路
  19. python脚本中为什么要在目录前加一个r
  20. Spring &lt;context:annotation-config/&gt; 解说(转)

热门文章

  1. linux 查看gpu信息
  2. Faster, more memory efficient and more ordered dictionaries on PyPy
  3. netty5拆包问题解决实例
  4. vscode片段
  5. PAT 甲级 1144 The Missing Number (20 分)(简单,最后一个测试点没过由于开的数组没必要大于N)
  6. SpringBoot 获取配置 @Value
  7. 【翻译】生成 Timestamps / Watermarks
  8. 比较两个数组的键名,并返回交集:array_intersect_key
  9. 高级UI-TableLayout
  10. python虚拟环境的配置: virtualenv 和 virtualenvwrapper-win 的用法