Impala 安装笔记2一hive和mysql安装
l 安装hive,hive-metastore hive-server
$ sudo yum install hive hive-metastore hive-server
l 安装mysql-server
$ sudo yum install mysql-server
l 启动mysql守护进程
$ sudo service mysqld start
l 下载mysql java驱动并复制到hive的lib中
$ cd ~
$ wget 'http://cdn.mysql.com/Downloads/Connector-J/mysql-connector-java-5.1.25.tar.gz'
$ tar xzf mysql-connector-java-5.1.25.tar.gz
$ sudo cp mysql-connector-java-5.1.25/mysql-connector-java-5.1.25-bin.jar
/usr/lib/hive/lib/
l
Mysql安全配置启动
$
sudo /usr/bin/mysql_secure_installation
[...]
Enter current password for root (enter for none):
OK, successfully used password, moving on...
[...]
Set root password? [Y/n] y
New password:hadoophive
Re-enter new password:hadoophive
Remove anonymous users? [Y/n] Y
[...]
Disallow root login remotely? [Y/n] N
[...]
Remove test database and access to it [Y/n] Y
[...]
Reload privilege tables now? [Y/n] Y
All done!
mysql作为hive的元数据存储数据库
在mysql中建立相应的元数据数据库,执行相应的sql脚本,并且建立供hive连接mysql的用户,赋予其权限。
$ mysql -u root –p chengyeliang
mysql> CREATE DATABASE metastore;
mysql> USE metastore;
mysql> SOURCE /usr/lib/hive/scripts/metastore/upgrade/mysql/hive-schema-0.10.0.mysql.sql; mysql> CREATE USER 'hive'@'%' IDENTIFIED BY 'chengyeliang';
mysql> CREATE USER 'hive'@'localhost' IDENTIFIED BY 'chengyeliang';
mysql> REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hive'@'%';
mysql> REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hive'@'localhost';
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,LOCK TABLES,EXECUTE ON metastore.* TO 'hive'@'%';
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,LOCK TABLES,EXECUTE ON metastore.* TO 'hive'@'localhost';
mysql> FLUSH PRIVILEGES;
mysql> quit;
修改hive的配置文件
$ sudo vim /etc/hive/conf/hive-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://10.127.130.199:3306/metastore?useUnicode=true&characterEn
coding=UTF-8</value>
<description>JDBC connect string for a JDBC metastore</description>
</property> <property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property> <property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>chengyeliang</value>
<description>password to use against metastore database</description>
</property> <property>
<name>localhost.job.tracker</name>
<value>localhost:8031</value>
</property> <property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://localhost:9083</value>
</property>
<property>
<name>hive.metastore.client.socket.timeout</name>
<value>3600</value>
<description>MetaStore Client socket timeout in seconds</description>
</property>
</configuration>
启动hive-metastore进程
$ sudo service hive-metastore start
或者后台启动服务:nohup hive --service metastore&
创建Hive所需的HDFS目录
$ sudo -u hdfs hadoop fs -mkdir /user/hive
$ sudo -u hdfs hadoop fs -chown hive /user/hive
$ sudo -u hdfs hadoop fs -mkdir /tmp
$ sudo -u hdfs hadoop fs -chmod 777 /tmp
$ sudo -u hdfs hadoop fs -chmod o+t /tmp
$ sudo -u hdfs hadoop fs -mkdir /data
$ sudo -u hdfs hadoop fs -chown hdfs /data
$ sudo -u hdfs hadoop fs -chmod 777 /data
$ sudo -u hdfs hadoop fs -chmod o+t /data
$ sudo chown -R hive:hive
/var/lib/hive
准备10万行记录的txt文件作为待查询的数据。
sudo vim /tmp/kv1.txt
建立一张新表
hive> CREATE TABLE IF NOT EXISTS chengyeliang ( foo INT,bar STRING ) ROW FORMAT DELIMITED FIELDS TERMINATED BY "\t" LINES TERMINATED BY "\n";
Time taken: 0.415 seconds
导入数据到table—chengyeliang
hive> LOAD DATA LOCAL INPATH '/tmp/kv1.txt' OVERWRITE INTO TABLE chengyeliang;
Time taken: 1.681 seconds
上面的路径要换成自己文本文件在hdfs上的绝对路径
如果上述操作ok,则安装成功。
最新文章
- C语言-指针
- 端口限制情况下php+xdebug环境配置
- 在Windows Server 2012 中安装 .NET 3.5 Framework
- 史上最全的css hack(ie6-9,firefox,chrome,opera,safari) (zz)
- JSP文件下载及出现getOutputStream() has already been called for this response的解决方法
- 转载---SQL Server XML基础学习<;2>;之--FOR XML AUTO/RAW
- Mac 下纯lua(三)
- SQL中如何使用UPDATE语句进行联表更新(转)
- BZOJ 1024 SCOI 2009 生日快乐 深搜
- mac地址静态捆绑,防止arp欺骗
- 关于ubuntu服务器上部署postgresql 以及安装pgadmin4管理工具(web版)
- 基于gitHub+hexo搭建的个人博客
- MongoDb安装--yum安装
- linux下64位汇编的系统调用(4)
- top命令用法详解
- word20161229
- LeetCode算法扫题系列83
- i.s.h.med Enhancement for cancelling an appointment
- C#启动另一个应用程序并传参数
- UWP 下载文件显示下载进度
热门文章
- 首次远程安装 GlassFish 后以远程 Web 方式访问其后台管理系统出现错误的解决方法(修订)
- AC日记——Pupils Redistribution Codeforces 779a
- POJ 3041 Asteroids 二分图
- CTSC2017游记&;心得记
- 第5章 Spring Boot 功能
- ios 6.0模拟器页面调出pop窗口消失后无法使用键盘
- 邁向IT專家成功之路的三十則鐵律 鐵律二十七 IT人夢想之道-實踐
- VC 读取服务器上的文件(HTTP方式) [转]
- 第六讲_图像分割Image Segmentation
- 转: 浅析Fusion-IO和Intel SSD