《OD大数据实战》Oozie环境搭建
一、Oozie环境搭建
1. 下载oozie-4.0.0-cdh5.3.6.tar.gz
下载地址:http://archive.cloudera.com/cdh5/cdh/5/
2. 解压
tar -zxvf oozie-4.0.0-cdh5.3.6.tar.gz -C /opt/modules/cdh/
3. 修改配置
1)cd /opt/modules/cdh/oozie-4.0.0-cdh5.3.6
mkdir libext
tar -zxvf oozie-hadooplibs-4.0.0-cdh5.3.6.tar.gz -C /tmp/
cd /opt/modules/cdh/oozie-4.0.0-cdh5.3.6/libext
cp -R /tmp/oozie-4.0.0-cdh5.3.6/hadooplibs/hadooplib-2.5.0-cdh5.3.6.oozie-4.0.0-cdh5.3.6/* ./
即在libext中添加hadooplib的jar包依赖
2)下载ext-2.2.zip
下载地址 http://download.csdn.net/detail/farawayplace613/708463
上传到linux中,
cp ext-2.2.zip /opt/modules/cdh/oozie-4.0.0-cdh5.3.6/libext
3)复制mysql jdbc连接驱动包到libext
mysql-connector-java-5.1.27-bin.jar
cp mysql-connector-java-5.1.-bin.jar /opt/modules/cdh/oozie-4.0.-cdh5.3.6/libext
4)修改hadoop的配置
cd /opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/etc/hadoop
vi core-site.xml
<!-- OOZIE -->
<property>
<name>hadoop.proxyuser.beifeng.hosts</name>
<value>beifeng-hadoop-02</value>
</property>
<property>
<name>hadoop.proxyuser.beifeng.groups</name>
<value>*</value>
</property>
5)创建数据库
在mysql里创建一个数据库,名称为oozie
create database oozie default character set utf8;
6)修改oozie-site.xml
<property>
<name>oozie.service.JPAService.jdbc.url</name>
<value>jdbc:mysql://beifeng-hadoop-02:3306/oozie</value>
<description>
JDBC URL.
</description>
</property> <property>
<name>oozie.service.JPAService.jdbc.username</name>
<value>root</value>
<description>
DB user name.
</description>
</property> <property>
<name>oozie.service.JPAService.jdbc.password</name>
<value>beifeng</value>
<description>
DB user password. IMPORTANT: if password is emtpy leave a space string, the service trims the value,
if empty Configuration assumes it is NULL.
</description>
</property>
7)指定所连接的hadoop集群的配置文件目录
<property>
<name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
<value>*=/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/etc/hadoop</value>
<description>
Comma separated AUTHORITY=HADOOP_CONF_DIR, where AUTHORITY is the HOST:PORT of
the Hadoop service (JobTracker, HDFS). The wildcard '*' configuration is
used when there is no exact match for an authority. The HADOOP_CONF_DIR contains
the relevant Hadoop *-site.xml files. If the path is relative is looked within
the Oozie configuration directory; though the path can be absolute (i.e. to point
to Hadoop client conf/ directories in the local filesystem.
</description>
</property>
8)打war包
bin/oozie-setup.sh prepare-war
9)将oozie运行过程中需要的依赖包共享到hdfs上
bin/oozie-setup.sh sharelib create -fs hdfs://beifeng-hadoop-02:9000 -locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz
10)初始化mysql oozie库
bin/oozie-setup.sh db create-run oozie.sql
11)启动检测运行
bin/oozied.sh start
ps -ef | grep oozie
netstat -tlnup | grep 11000
12)访问web ui
http://beifeng-hadoop-02:11000/oozie/
最新文章
- java中File类的getPath(),getAbsolutePath(),getCanonicalPath()区别
- 数学软件 之 基于MATLAB的DFP算法
- Linux常用压缩和解压命令
- .ascx.g.cs文件不能生成 The name ‘InitializeControl’ does not exist in the current context - Visual Web part Sharepoint
- Editor Scripting学习笔记之Menu Item
- VS2005打开VS2008 VS2010 VS2012
- Android手机配置gcc,实现手机编译代码
- BZOJ 3589 动态树 树链拆分+纳入和排除定理
- UML基础概念
- 日期时间插件flatpickr.js使用方法
- Java 9 揭秘(1.Java入门介绍)
- IT小天博客APP版本
- [Swift]LeetCode446. 等差数列划分 II - 子序列 | Arithmetic Slices II - Subsequence
- java HttpServletRequest 重复流读取
- python中前后端通信方法Ajax和ORM映射(form表单提交)
- Ocelot 使用
- 第八届蓝桥杯国赛java B组第三题
- vivado第一天从建立文件运行小程序开始
- angular 路由
- 【云计算】pig基础、运行、编写