CHD-5.3.6集群上hive安装

解压过后：

[hadoop@master CDH5.3.6]$ ls -rlt
total 8
drwxr-xr-x. 17 hadoop hadoop 4096 Jun 2 16:07 hadoop-2.5.0-cdh5.3.6
drwxr-xr-x. 11 hadoop hadoop 4096 Jun 2 16:28 hive-0.13.1-cdh5.3.6

1.配置hive-env.sh

export JAVA_HOME=/usr/local/jdk1.

export HADOOP_HOME=/home/hadoop/CDH5.3.6/hadoop-2.5.-cdh5.3.6

export HIVE_HOME=/home/hadoop/CDH5.3.6/hive-0.13.-cdh5.3.6

export HIVE_CONF_DIR=/home/hadoop/CDH5.3.6/hive-0.13.-cdh5.3.6/conf

2.配置hive-log4j.properties

hive.log.dir=/home/hadoop/CDH5.3.6/hive-0.13.-cdh5.3.6/log

3.配置hive-site.xml

这个寻找Apache-hadoop下的就可以，直接考过来就可以，在conf 目录下

4.配置环境变量

vi .bash_profile

export HADOOP_HOME=/home/hadoop/CDH5.3.6/hadoop-2.5.-cdh5.3.6

export HADOOP_MAPRED_HOME=$HADOOP_HOME

export HADOOP_COMMON_HOME=$HADOOP_HOME

export HADOOP_HDFS_HOME=$HADOOP_HOME

export YARN_HOME=$HADOOP_HOME

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

export HIVE_HOME=/home/hadoop/CDH5.3.6/hive-0.13.-cdh5.3.6

export HADOOP_INSTALL=$HADOOP_HOME

export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin:$MAVEN_HOME/bin:$HIVE_HOME/bin

5.拷贝MySQL包

cp /home/hadoop/hive/lib/mysql-connector-java-5.1..jar ./

6.hive命令报错：

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
解决方法：

格式化MySQL：
schematool -dbType mysql -initSchema

7.进入hive

hive (default)>

              > CREATE TABLE dept(

              >   deptno int,

              >   dname string,

              >   loc string)

              > ROW FORMAT DELIMITED FIELDS TERMINATED BY ','  STORED AS textfile;

OK

Time taken: 0.57 seconds

8.准备数据：

vi detp.txt

,ACCOUNTING,NEW YORK

,RESEARCH,DALLAS

,SALES,CHICAGO

,OPERATIONS,BOSTON

9.装数据：

load data local inpath '/home/hadoop/tmp/detp.txt' overwrite into table dept;

10.查询：

hive (default)> select count(1) from dept;

Total jobs =

Launching Job  out of

Number of reduce tasks determined at compile time:

In order to change the average load for a reducer (in bytes):

  set hive.exec.reducers.bytes.per.reducer=<number>

In order to limit the maximum number of reducers:

  set hive.exec.reducers.max=<number>

In order to set a constant number of reducers:

  set mapreduce.job.reduces=<number>

Starting Job = job_1559517371869_0001, Tracking URL = http://master:8088/proxy/application_1559517371869_0001/

Kill Command = /home/hadoop/CDH5.3.6/hadoop-2.5.-cdh5.3.6/bin/hadoop job  -kill job_1559517371869_0001

Hadoop job information for Stage-: number of mappers: ; number of reducers:

-- ::, Stage- map = %,  reduce = %

-- ::, Stage- map = %,  reduce = %, Cumulative CPU 0.79 sec

-- ::, Stage- map = %,  reduce = %, Cumulative CPU 1.7 sec

MapReduce Total cumulative CPU time:  seconds  msec

Ended Job = job_1559517371869_0001

MapReduce Jobs Launched:

Stage-Stage-: Map:   Reduce:    Cumulative CPU: 1.7 sec   HDFS Read:  HDFS Write:  SUCCESS

Total MapReduce CPU Time Spent:  seconds  msec

OK

_c0

Time taken: 22.14 seconds, Fetched:  row(s)

页面验证：

http://192.168.1.30:8088/cluster

http://192.168.1.30:50070/dfshealth.html#tab-overview

巴特西

CHD-5.3.6集群上hive安装

最新文章

热门文章