一, MHA 介绍　　

　　MHA（Master High Availability）目前在MySQL高可用方面是一个相对成熟的解决方案，它由日本DeNA公司youshimaton（现就职于Facebook公司）开发，是一套优秀的作为MySQL高可用性环境下故障切换和主从提升的高可用软件。在MySQL故障切换过程中，MHA能做到在0~30秒之内自动完成数据库的故障切换操作，并且在进行故障切换的过程中，MHA能在最大程度上保证数据的一致性，以达到真正意义上的高可用。

　　该软件由两部分组成：MHA Manager（管理节点）和MHA Node（数据节点）。MHA Manager可以单独部署在一台独立的机器上管理多个master-slave集群，也可以部署在一台slave节点上。MHA Node运行在每台MySQL服务器上，MHA Manager会定时探测集群中的master节点，当master出现故障时，它可以自动将最新数据的slave提升为新的master，然后将所有其他的slave重新指向新的master。整个故障转移过程对应用程序完全透明。

　　在MHA自动故障切换过程中，MHA试图从宕机的主服务器上保存二进制日志，最大程度的保证数据的不丢失，但这并不总是可行的。例如，如果主服务器硬件故障或无法通过ssh访问，MHA没法保存二进制日志，只进行故障转移而丢失了最新的数据。使用MySQL 5.5的半同步复制，可以大大降低数据丢失的风险。MHA可以与半同步复制结合起来。如果只有一个slave已经收到了最新的二进制日志，MHA可以将最新的二进制日志应用于其他所有的slave服务器上，因此可以保证所有节点的数据一致性。

　　目前MHA主要支持一主多从的架构，要搭建MHA,要求一个复制集群中必须最少有三台数据库服务器，一主二从，即一台充当master，一台充当备用master，另外一台充当从库，因为至少需要三台服务器，出于机器成本的考虑，淘宝也在该基础上进行了改造，目前淘宝TMHA已经支持一主一从。

二, MHA搭建前

　　 MHA搭建前必须要完成一主多从的主从复制项目:

　　　　环境:

　　　　　　

db01

 主 10.0.0.51
db02 从 10.0.0.52
db03 从 10.0.0.53

　　　　三台节点配置:　

vim /etc/my.cnf

[mysqld]

basedir=/application/mysql

datadir=/application/mysql/data

socket=/tmp/mysql.sock

log-error=/var/log/mysql.log

log_bin=/data/mysql/mysql-bin

binlog_format=row

skip-name-resolve

server-id=51

gtid-mode=on

enforce-gtid-consistency=true

log-slave-updates=1

[client]

socket=/tmp/mysql.sock

slave1：10.0.0.52

vim /etc/my.cnf

[mysqld]

basedir=/application/mysql

datadir=/application/mysql/data

socket=/tmp/mysql.sock

log-error=/var/log/mysql.log

log_bin=/data/mysql/mysql-bin

binlog_format=row

skip-name-resolve

server-id=52

gtid-mode=on

enforce-gtid-consistency=true

log-slave-updates=1

[client]

socket=/tmp/mysql.sock

slave2：10.0.0.53

vim  /etc/my.cnf

[mysqld]

basedir=/application/mysql

datadir=/application/mysql/data

socket=/tmp/mysql.sock

log-error=/var/log/mysql.log

log_bin=/data/mysql/mysql-bin

binlog_format=row

skip-name-resolve

server-id=53

gtid-mode=on

enforce-gtid-consistency=true

log-slave-updates=1

[client]

socket=/tmp/mysql.sock

3.重新初始化三台机器数据

/application/mysql/scripts/mysql_install_db --basedir=/application/mysql/ --datadir=/application/mysql/data --user=mysql

4. 分别启动三台数据库服务器

/etc/init.d/mysqld start

三. MHA部署开始

　　　　构建GTID:　　　　　

51节点:

grant replication slave  on *.* to repl@'10.0.0.%' identified by '123';

52\53节点:

change master to master_host='10.0.0.51',master_user='repl',master_password='123' ,MASTER_AUTO_POSITION=1;

start slave;

　　　　密钥乱发:

　　　　　主节点配值:　　　　　　　

rm -rf /root/.ssh

ssh-keygen

cd /root/.ssh

mv id_rsa.pub authorized_keys

scp  -r  /root/.ssh  10.0.0.52

scp  -r  /root/.ssh  10.0.0.53

　　　　　　各个节点密钥完成无密码两两互联

51节点:

ssh 10.0.0.51 date

ssh 10.0.0.52 date

ssh 10.0.0.53 date

52节点:

ssh 10.0.0.51 date

ssh 10.0.0.52 date

ssh 10.0.0.53 date

53节点:

ssh 10.0.0.51 date

ssh 10.0.0.52 date

ssh 10.0.0.53 date

　　　　数据库配置:

51节点 52节点  53节点配置

set global relay_log_purge = 0; 

52节点  53节点配置

set global read_only=1;

　　　　服务器配置　

ln -s /application/mysql/bin/mysqlbinlog /usr/bin/mysqlbinlog

ln -s /application/mysql/bin/mysql /usr/bin/mysql

　　　　

　　　　软件下载:

　　　　　　

　　　　　　下载mha软件，mha官网：https://code.google.com/archive/p/mysql-master-ha/
 
　　　　

　　　　安装依赖包:

　　　　　　    yum install perl-DBD-MySQL -y

　　　　　　

　　　　安装node

　　　　　　所有节点

　　　　　　rpm -ivh mha4mysql-node-0.57-0.el7.noarch.rpm

　　　　

　　　　主库创建MHA用户

　　　　　　　grant all privileges on *.* to mha@'10.0.0.%' identified by 'mha';

　　　　

　　　　53节点配置MHA

　　　　　　　 yum install -y perl-Config-Tiny epel-release perl-Log-Dispatch perl-Parallel-ForkManager perl-Time-HiRes

　　　　　　　rpm -ivh mha4mysql-manager-0.57-0.el7.noarch.rpm

　　

　　　　53节点配置文件配置　　　　　

#创建配置文件目录

 mkdir -p /etc/mha

#创建日志目录

 mkdir -p /var/log/mha/app1

#编辑mha配置文件

　
vim /etc/mha/app1.cnf

[server default]

manager_log=/var/log/mha/app1/manager

manager_workdir=/var/log/mha/app1

master_binlog_dir=/data/mysql/

user=mha

password=mha

ping_interval=2

repl_password=123

repl_user=repl

ssh_user=root                               

[server1]

hostname=10.0.0.51

port=3306                                  

[server2]

hostname=10.0.0.52

port=3306

[server3]

hostname=10.0.0.53

port=3306

　　

　　　　MHA 检查:

　　　　　　

masterha_check_ssh  --conf=/etc/mha/app1.cnf

　　　　

　　　　全部ok为正常

　　　　

　　　　主从状态:

　　　　

masterha_check_repl  --conf=/etc/mha/app1.cnf

　　　　

　　　　

　　　　启动MHA

nohup masterha_manager --conf=/etc/mha/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/mha/app1/manager.log 2>&1 &

　　　　

四, 故障演练

　　　　某一天主库宕机了

　　　　　　pkill mysqld

　　　　同时MHA架构日志:　　

 Mon Nov  ::  - [warning] Got error on MySQL select ping:  (MySQL server has gone away)

 Mon Nov  ::  - [info] Executing SSH check script: exit

 Mon Nov  ::  - [warning] Got error on MySQL connect:  (Can't connect to MySQL server on '10.0.0.51' (111))

 Mon Nov 26 17:31:12 2018 - [warning] Connection failed 2 time(s)..

 Mon Nov 26 17:31:14 2018 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '10.0.0.51' ())

 Mon Nov  ::  - [warning] Connection failed  time(s)..

 Mon Nov  ::  - [warning] HealthCheck: Got timeout on checking SSH connection to 10.0.0.51! at /usr/share/perl5/vendor_perl/MHA/HealthCheck.pm line .

 Mon Nov  ::  - [warning] Got error on MySQL connect:  (Can't connect to MySQL server on '10.0.0.51' (111))

 Mon Nov 26 17:31:16 2018 - [warning] Connection failed 4 time(s)..

 Mon Nov 26 17:31:16 2018 - [warning] Master is not reachable from health checker!

 Mon Nov 26 17:31:16 2018 - [warning] Master 10.0.0.51(10.0.0.51:3306) is not reachable!

 Mon Nov 26 17:31:16 2018 - [warning] SSH is NOT reachable.

 Mon Nov 26 17:31:16 2018 - [info] Connecting to a master server failed. Reading configuration file /etc/masterha_default.cnf and /etc/mha/app1.cnf again, and trying to connect to all servers to check server status..

 Mon Nov 26 17:31:16 2018 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.

 Mon Nov 26 17:31:16 2018 - [info] Reading application default configuration from /etc/mha/app1.cnf..

 Mon Nov 26 17:31:16 2018 - [info] Reading server configuration from /etc/mha/app1.cnf..

 Mon Nov 26 17:31:17 2018 - [info] GTID failover mode = 1

 Mon Nov 26 17:31:17 2018 - [info] Dead Servers:

 Mon Nov 26 17:31:17 2018 - [info]   10.0.0.51(10.0.0.51:3306)

 Mon Nov 26 17:31:17 2018 - [info] Alive Servers:

 Mon Nov 26 17:31:17 2018 - [info]   10.0.0.52(10.0.0.52:3306)

 Mon Nov 26 17:31:17 2018 - [info]   10.0.0.53(10.0.0.53:3306)

 Mon Nov 26 17:31:17 2018 - [info] Alive Slaves:

 Mon Nov 26 17:31:17 2018 - [info]   10.0.0.52(10.0.0.52:3306)  Version=5.6.38-log (oldestmajor version between slaves) log-bin:enabled

 Mon Nov 26 17:31:17 2018 - [info]     GTID ON

 Mon Nov 26 17:31:17 2018 - [info]     Replicating from 10.0.0.51(10.0.0.51:3306)

 Mon Nov 26 17:31:17 2018 - [info]   10.0.0.53(10.0.0.53:3306)  Version=5.6.38-log (oldestmajor version between slaves) log-bin:enabled

 Mon Nov 26 17:31:17 2018 - [info]     GTID ON

 Mon Nov 26 17:31:17 2018 - [info]     Replicating from 10.0.0.51(10.0.0.51:3306)

 Mon Nov 26 17:31:17 2018 - [info] Checking slave configurations..

 Mon Nov 26 17:31:17 2018 - [info] Checking replication filtering settings..

 Mon Nov 26 17:31:17 2018 - [info]  Replication filtering check ok.

 Mon Nov 26 17:31:17 2018 - [info] Master is down!

 Mon Nov 26 17:31:17 2018 - [info] Terminating monitoring script.

 Mon Nov 26 17:31:17 2018 - [info] Got exit code 20 (Master dead).

 Mon Nov 26 17:31:17 2018 - [info] MHA::MasterFailover version 0.57.

 Mon Nov 26 17:31:17 2018 - [info] Starting master failover.

 Mon Nov 26 17:31:17 2018 - [info]

 Mon Nov 26 17:31:17 2018 - [info] * Phase 1: Configuration Check Phase..

 Mon Nov 26 17:31:17 2018 - [info]

 Mon Nov 26 17:31:18 2018 - [info] GTID failover mode = 1

 Mon Nov 26 17:31:18 2018 - [info] Dead Servers:

 Mon Nov 26 17:31:18 2018 - [info]   10.0.0.51(10.0.0.51:3306)

 Mon Nov 26 17:31:18 2018 - [info] Checking master reachability via MySQL(double check)...

 Mon Nov 26 17:31:18 2018 - [info]  ok.

 Mon Nov 26 17:31:18 2018 - [info] Alive Servers:

 Mon Nov 26 17:31:18 2018 - [info]   10.0.0.52(10.0.0.52:3306)

 Mon Nov 26 17:31:18 2018 - [info]   10.0.0.53(10.0.0.53:3306)

 Mon Nov 26 17:31:18 2018 - [info] Alive Slaves:

 Mon Nov 26 17:31:18 2018 - [info]   10.0.0.52(10.0.0.52:3306)  Version=5.6.38-log (oldestmajor version between slaves) log-bin:enabled

 Mon Nov 26 17:31:18 2018 - [info]     GTID ON

 Mon Nov 26 17:31:18 2018 - [info]     Replicating from 10.0.0.51(10.0.0.51:3306)

 Mon Nov 26 17:31:18 2018 - [info]   10.0.0.53(10.0.0.53:3306)  Version=5.6.38-log (oldestmajor version between slaves) log-bin:enabled

 Mon Nov 26 17:31:18 2018 - [info]     GTID ON

 Mon Nov 26 17:31:18 2018 - [info]     Replicating from 10.0.0.51(10.0.0.51:3306)

 Mon Nov 26 17:31:18 2018 - [info] Starting GTID based failover.

 Mon Nov 26 17:31:18 2018 - [info]

 Mon Nov 26 17:31:18 2018 - [info] ** Phase 1: Configuration Check Phase completed.

 Mon Nov 26 17:31:18 2018 - [info]

 Mon Nov 26 17:31:18 2018 - [info] * Phase 2: Dead Master Shutdown Phase..

 Mon Nov 26 17:31:18 2018 - [info]

 Mon Nov 26 17:31:18 2018 - [info] Forcing shutdown so that applications never connect to the current master..

 Mon Nov 26 17:31:18 2018 - [warning] master_ip_failover_script is not set. Skipping invalidating dead master IP address.

 Mon Nov 26 17:31:18 2018 - [warning] shutdown_script is not set. Skipping explicit shutting down of the dead master.

 Mon Nov 26 17:31:19 2018 - [info] * Phase 2: Dead Master Shutdown Phase completed.

 Mon Nov 26 17:31:19 2018 - [info]

 Mon Nov 26 17:31:19 2018 - [info] * Phase 3: Master Recovery Phase..

 Mon Nov 26 17:31:19 2018 - [info]

 Mon Nov 26 17:31:19 2018 - [info] * Phase 3.1: Getting Latest Slaves Phase..

 Mon Nov 26 17:31:19 2018 - [info]

 Mon Nov 26 17:31:19 2018 - [info] The latest binary log file/position on all slaves is mysql-bin.000003:792

 Mon Nov 26 17:31:19 2018 - [info] Retrieved Gtid Set: b54af3fb-f133-11e8-825d-00505628e54f:1-3

 Mon Nov 26 17:31:19 2018 - [info] Latest slaves (Slaves that received relay log files to the latest):

 Mon Nov 26 17:31:19 2018 - [info]   10.0.0.52(10.0.0.52:3306)  Version=5.6.38-log (oldestmajor version between slaves) log-bin:enabled

 Mon Nov 26 17:31:19 2018 - [info]     GTID ON

 Mon Nov 26 17:31:19 2018 - [info]     Replicating from 10.0.0.51(10.0.0.51:3306)

 Mon Nov 26 17:31:19 2018 - [info]   10.0.0.53(10.0.0.53:3306)  Version=5.6.38-log (oldestmajor version between slaves) log-bin:enabled

 Mon Nov 26 17:31:19 2018 - [info]     GTID ON

 Mon Nov 26 17:31:19 2018 - [info]     Replicating from 10.0.0.51(10.0.0.51:3306)

 Mon Nov 26 17:31:19 2018 - [info] The oldest binary log file/position on all slaves is mysql-bin.000003:792

 Mon Nov 26 17:31:19 2018 - [info] Retrieved Gtid Set: b54af3fb-f133-11e8-825d-00505628e54f:1-3

 Mon Nov 26 17:31:19 2018 - [info] Oldest slaves:

 Mon Nov 26 17:31:19 2018 - [info]   10.0.0.52(10.0.0.52:3306)  Version=5.6.38-log (oldestmajor version between slaves) log-bin:enabled

 Mon Nov 26 17:31:19 2018 - [info]     GTID ON

 Mon Nov 26 17:31:19 2018 - [info]     Replicating from 10.0.0.51(10.0.0.51:3306)

 Mon Nov 26 17:31:19 2018 - [info]   10.0.0.53(10.0.0.53:3306)  Version=5.6.38-log (oldestmajor version between slaves) log-bin:enabled

 Mon Nov 26 17:31:19 2018 - [info]     GTID ON

 Mon Nov 26 17:31:19 2018 - [info]     Replicating from 10.0.0.51(10.0.0.51:3306)

 Mon Nov 26 17:31:19 2018 - [info]

 Mon Nov 26 17:31:19 2018 - [info] * Phase 3.3: Determining New Master Phase..

 Mon Nov 26 17:31:19 2018 - [info]

 Mon Nov 26 17:31:19 2018 - [info] Searching new master from slaves..

 Mon Nov 26 17:31:19 2018 - [info]  Candidate masters from the configuration file:

 Mon Nov 26 17:31:19 2018 - [info]  Non-candidate masters:

 Mon Nov 26 17:31:19 2018 - [info] New master is 10.0.0.52(10.0.0.52:3306)

 Mon Nov 26 17:31:19 2018 - [info] Starting master failover..

 Mon Nov 26 17:31:19 2018 - [info]

 From:

 10.0.0.51(10.0.0.51:3306) (current master)

  +--10.0.0.52(10.0.0.52:3306)

  +--10.0.0.53(10.0.0.53:3306)

 To:

 10.0.0.52(10.0.0.52:3306) (new master)

  +--10.0.0.53(10.0.0.53:3306)

 Mon Nov 26 17:31:19 2018 - [info]

 Mon Nov 26 17:31:19 2018 - [info] * Phase 3.3: New Master Recovery Phase..

 Mon Nov 26 17:31:19 2018 - [info]

 Mon Nov 26 17:31:19 2018 - [info]  Waiting all logs to be applied..

 Mon Nov 26 17:31:19 2018 - [info]   done.

 Mon Nov 26 17:31:19 2018 - [info] Getting new master's binlog name and position..

 Mon Nov  ::  - [info]  mysql-bin.:

 Mon Nov  ::  - [info]  All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='10.0.0.52', MASTER_PORT=, MASTER_AUTO_POSITION=, MASTER_USER='repl', MASTER_PASSWORD='xxx';

 Mon Nov  ::  - [info] Master Recovery succeeded. File:Pos:Exec_Gtid_Set: mysql-bin., , b54af3fb-f133-11e8-825d-00505628e54f:-

 Mon Nov  ::  - [warning] master_ip_failover_script is not set. Skipping taking over new master IP address.

 Mon Nov  ::  - [info] Setting read_only= on 10.0.0.52(10.0.0.52:)..

 Mon Nov  ::  - [info]  ok.

 Mon Nov  ::  - [info] ** Finished master recovery successfully.

 Mon Nov  ::  - [info] * Phase : Master Recovery Phase completed.

 Mon Nov  ::  - [info]

 Mon Nov  ::  - [info] * Phase : Slaves Recovery Phase..

 Mon Nov  ::  - [info]

 Mon Nov  ::  - [info]

 Mon Nov  ::  - [info] * Phase 4.1: Starting Slaves in parallel..

 Mon Nov  ::  - [info]

 Mon Nov  ::  - [info] -- Slave recovery on host 10.0.0.53(10.0.0.53:) started, pid: . Check tmp log /var/log/mha/app1/10.0..53_3306_20181126173117.log if it takes time..

 Mon Nov  ::  - [info]

 Mon Nov  ::  - [info] Log messages from 10.0.0.53 ...

 Mon Nov  ::  - [info]

 Mon Nov  ::  - [info]  Resetting slave 10.0.0.53(10.0.0.53:) and startingreplication from the new master 10.0.0.52(10.0.0.52:)..

 Mon Nov  ::  - [info]  Executed CHANGE MASTER.

 Mon Nov  ::  - [info]  Slave started.

 Mon Nov  ::  - [info]  gtid_wait(b54af3fb-f133-11e8-825d-00505628e54f:-) completed on 10.0.0.53(10.0.0.53:). Executed  events.

 Mon Nov  ::  - [info] End of log messages from 10.0.0.53.

 Mon Nov  ::  - [info] -- Slave on host 10.0.0.53(10.0.0.53:) started.

 Mon Nov  ::  - [info] All new slave servers recovered successfully.

 Mon Nov  ::  - [info]

 Mon Nov  ::  - [info] * Phase : New master cleanup phase..

 Mon Nov  ::  - [info]

 Mon Nov  ::  - [info] Resetting slave info on the new master..

 Mon Nov  ::  - [info]  10.0.0.52: Resetting slave info succeeded.

 Mon Nov  ::  - [info] Master failover to 10.0.0.52(10.0.0.52:) completed successfully.

 Mon Nov  ::  - [info] Deleted server1 entry from /etc/mha/app1.cnf .

 Mon Nov  ::  - [info]

 ----- Failover Report -----

 app1: MySQL Master failover 10.0.0.51(10.0.0.51:) to 10.0.0.52(10.0.0.52:) succeeded

 Master 10.0.0.51(10.0.0.51:) is down!

 Check MHA Manager logs at db03:/var/log/mha/app1/manager for details.

 Started automated(non-interactive) failover.

 Selected 10.0.0.52(10.0.0.52:) as a new master.

 10.0.0.52(10.0.0.52:): OK: Applying all logs succeeded.

 10.0.0.53(10.0.0.53:): OK: Slave started, replicating from 10.0.0.52(10.0.0.52:)

 10.0.0.52(10.0.0.52:): Resetting slave info succeeded.

 Master failover to 10.0.0.52(10.0.0.52:) completed successfully.

　　　　可以发现主库实现漂移自动转到53节点

　　　　

　　　　恢复机器:

　　

　　　　再MHA的日志这里可以看见他说明了要恢复服务器的内容

　　　　

　　　　而且

　　　　　　vim /etc/mha/app1.cnf

　　　　　　配置文件中不需要的服务器自动删除

　　　　　　重新启动服务器的话需要把配置文件重新配置

 [server default]

 manager_log=/var/log/mha/app1/manager

 manager_workdir=/var/log/mha/app1

 master_binlog_dir=/data/mysql/

 password=mha

 ping_interval=

 repl_password=

 repl_user=repl

 ssh_user=root

 user=mha

 [server1]

 hostname=10.0.0.51

 port=

 [server2]

 hostname=10.0.0.52

 port=

 [server3]

 hostname=10.0.0.53

 port=

　　　　再宕机的服务器重新配置slave　　　　　

CHANGE MASTER TO MASTER_HOST='10.0.0.52', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='123';

start slave;

　　　　

　　　　

　　　　MHA重新启动

　　　　　　重新检测:

[root@db03 ~]# masterha_check_ssh  --conf=/etc/mha/app1.cnf

Mon Nov  ::  - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.

Mon Nov  ::  - [info] Reading application default configuration from /etc/mha/app1.cnf..

Mon Nov  ::  - [info] Reading server configuration from /etc/mha/app1.cnf..

Mon Nov  ::  - [info] Starting SSH connection tests..

Mon Nov  ::  - [debug]

Mon Nov  ::  - [debug]  Connecting via SSH from root@10.0.0.51(10.0.0.51:) to root@10.0.0.52(10.0.0.52:)..

Mon Nov  ::  - [debug]   ok.

Mon Nov  ::  - [debug]  Connecting via SSH from root@10.0.0.51(10.0.0.51:) to root@10.0.0.53(10.0.0.53:)..

Mon Nov  ::  - [debug]   ok.

Mon Nov  ::  - [debug]

Mon Nov  ::  - [debug]  Connecting via SSH from root@10.0.0.52(10.0.0.52:) to root@10.0.0.51(10.0.0.51:)..

Mon Nov  ::  - [debug]   ok.

Mon Nov  ::  - [debug]  Connecting via SSH from root@10.0.0.52(10.0.0.52:) to root@10.0.0.53(10.0.0.53:)..

Mon Nov  ::  - [debug]   ok.

Mon Nov  ::  - [debug]

Mon Nov  ::  - [debug]  Connecting via SSH from root@10.0.0.53(10.0.0.53:) to root@10.0.0.51(10.0.0.51:)..

Mon Nov  ::  - [debug]   ok.

Mon Nov  ::  - [debug]  Connecting via SSH from root@10.0.0.53(10.0.0.53:) to root@10.0.0.52(10.0.0.52:)..

Mon Nov  ::  - [debug]   ok.

Mon Nov  ::  - [info] All SSH connection tests passed successfully.

[root@db03 ~]#
nohup masterha_manager --conf=/etc/mha/app1.cnf --remove_dead_master_conf --ignore_last_failover< /dev/null > /var/log/mha/app1/manager.log >& &

　　　　

　　　　

最新文章

  1. 利用ThinkPHP自带的七牛云驱动上传文件到七牛云以及删除七牛云文件方法
  2. ServiceManager: Permmission failure: android.permission.RECORD_AUDIO
  3. 二分图水一波~~~~d带你飞
  4. jQuery 基本过滤选择器注意点
  5. 无法远程连接ubuntu下的mysql
  6. 【COGS &amp; USACO Training】710. 命名那个数字（hash+水题+dfs）
  7. vs2012 Nuget错误：“未能解析此远程名称api.nuget.org”
  8. 使用PHP抓取网站ico图标
  9. MSCRM4.0如何使js事件在批量编辑表单中触发
  10. CodeForces 22D Segments 排序水问题
  11. 2017中国大学生程序设计竞赛 - 网络选拔赛 HDU 6155 Subsequence Count 矩阵快速幂
  12. (Matlab)GPU计算及CPU计算能力的比较
  13. 《转》优化UITableViewCell高度计算的那些事
  14. docker~aspnetcore2.0镜像缺少libgdiplus问题
  15. redis数据库安装 redis持久化及主从复制
  16. Oracle 用户权限 Grant
  17. Skyline TerraExplorer -二次开发- 加载外部数据的各种连接串
  18. Django+Xadmin打造在线教育系统(九)
  19. Java语法之注解
  20. 一本通1548【例 2】A Simple Problem with Integers

热门文章

  1. javascript总结11:JavaScript的自增自减
  2. HDU 3362 Fix （状压DP）
  3. （转）每位设计师都应该拥有的50个CSS代码片段
  4. GlobalAlloc()和malloc()、HeapAlloc()
  5. jQuery高级
  6. react-native-echarts构建的图表出现滚动条并且可以滑动的问题
  7. Go语言最佳实践——异常和错误
  8. Django中使用后台网站模板
  9. 基于zookeeper实现高性能分布式锁
  10. centos7下git的安装和配置