这里使用的版本号是cdh发行的pig-0.12.0-cdh5.1.2 下载地址点这里

1.Pig简单介绍:

Pig是yahoo捐献给apache的一个项目。它是SQL-like语言。是在MapReduce上构建的一种高级查询语言,把一些运算编译进MapReduce模型的Map和Reduce中,而且用户能够定义自己的功能。这是Yahoo开发的又一个克隆Google的项目:Sawzall。

Pig是一个client应用程序,就算你要在Hadoop集群上执行Pig。也不须要在集群上装额外的东西

2.安装

解压下载完毕的pig到指定文件夹。我这里将其解压到用户hadoop文件夹下

<span style="font-size:18px;">hadoop@caozw:~/pig/conf$ tar -xzvf ~/Downloads/pig-0.12.0-cdh5.1.2.tar.gz  -C ~/
</span>

为配置方便 这里将其建立软链接到pig

<span style="font-size:18px;">hadoop@caozw:~/pig/conf$ ln -s pig-0.12.0-cdh5.1.2/ pig</span>

3.环境变量配置

通过编辑/etc/.profile文件或者是用户文件夹以下的~/.profile文件。我这里编辑hadoop用户文件夹以下的配置文件来配置

<span style="font-size:18px;">export PIG_HOME=/home/hadoop/pig
export PIG_CLASSPATH=${HADOOP_HOME}/etc/hadoop
export PATH=$PATH:$PIG_HOME/bin</span>

当中PIG_CLASSPATH指定了hadoop的配置文件路径。本地模式不要配置,假设须要訪问hadoop的时候必须配置

通过source ~/.profile使配置生效

4.local执行

<span style="font-size:18px;">hadoop@caozw:~/pig/conf$ pig -x local
2014-10-13 19:17:34,862 [main] INFO org.apache.pig.Main - Apache Pig version 0.12.0-cdh5.1.2 (rexported) compiled Aug 25 2014, 19:51:48
2014-10-13 19:17:34,863 [main] INFO org.apache.pig.Main - Logging error messages to: /home/hadoop/pig-0.12.0-cdh5.1.2/conf/pig_1413199054861.log
2014-10-13 19:17:34,905 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/hadoop/.pigbootup not found
2014-10-13 19:17:35,204 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-10-13 19:17:35,205 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-10-13 19:17:35,206 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.3.0-cdh5.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-0.98.1-cdh5.1.2/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2014-10-13 19:17:35,732 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2014-10-13 19:17:35,918 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2014-10-13 19:17:35,922 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
grunt> </span>

出现grunt提示说明启动成功

5.hadoop执行

须要启动hadoop集群,pig会依据PIG_CLASSPATH的路径以下的配置文件自己主动识别hadoop集群

<span style="font-size:18px;">grunt> hadoop@caozw:~/pig/conf$ pig
2014-10-13 19:18:36,511 [main] INFO org.apache.pig.Main - Apache Pig version 0.12.0-cdh5.1.2 (rexported) compiled Aug 25 2014, 19:51:48
2014-10-13 19:18:36,511 [main] INFO org.apache.pig.Main - Logging error messages to: /home/hadoop/pig-0.12.0-cdh5.1.2/conf/pig_1413199116510.log
2014-10-13 19:18:36,541 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/hadoop/.pigbootup not found
2014-10-13 19:18:36,849 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-10-13 19:18:36,849 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-10-13 19:18:36,849 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://192.168.118.168:9100
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.3.0-cdh5.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-0.98.1-cdh5.1.2/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2014-10-13 19:18:37,071 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2014-10-13 19:18:38,379 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
grunt> </span>

至此安装已经完毕了。安装非常easy可是功能不简单,使用会在后面一步步展开。而且能够使用pig为hdfs的数据建索引并推送到elasticsearch集群中。非常期待~

最新文章

  1. Curator框架的使用
  2. appium for iOS config
  3. Memory Management in Open Cascade
  4. BZOJ2706 : [SDOI2012]棋盘覆盖
  5. [ACM训练] DEV C++如何处理不能调试
  6. 动态链接库(DLL)总结
  7. npoi实现 从固定的行读取数据作为表头并返回datable
  8. 返回顶部(解决IE6固定定位)
  9. YCbCr;YUV;RGB
  10. linux驱动系列之arm汇编
  11. 未能从程序集“System.ServiceModel, Version=3.0.0.0”中加载类型“System.ServiceModel.Activation.HttpModule” 的解决办法
  12. 【Eclipse】修改java代码不强制重启
  13. 聊聊Spring Cloud版本的那些事儿
  14. 深入理解Zuul之源码解析
  15. magento开发手册之目录结构
  16. 分享一个android debug模式,出现 waiting for debugger把界面卡住,取巧的解决办法
  17. Go 语言中的 new() 和 make()的区别
  18. golang程序在windows上,注册为服务
  19. Smart Pointer 智能指针
  20. Pylons架构网站开发从0到1

热门文章

  1. python笔记四:面向对象
  2. 【动态规划】POJ1661 Help Jimmy
  3. [AGC027F]Grafting
  4. 【后缀自动机】hihocoder1445 后缀自动机二&#183;重复旋律5
  5. CDOJ 1281 暴兵的卿学姐 构造题
  6. 用asyncio的异步网络连接来获取sina、sohu和163的网站首页
  7. 让你的WPF程序在Win7下呈现Win8风格主题
  8. 最好的拖拽js
  9. datetimepicker只显示日期,不显示时分秒
  10. asp.net core 1.0初识