运行平台:Hadoop 2.6.3

模式:完全分布模式

1、准备统计文本,以一段文字为例:eg.txt

The Project Gutenberg EBook of War and Peace, by Leo Tolstoy

This eBook is for the use of anyone anywhere at no cost and with almost
no restrictions whatsoever. You may copy it, give it away or re-use it
under the terms of the Project Gutenberg License included with this
eBook or online at www.gutenberg.org Title: War and Peace Author: Leo Tolstoy

2、在Shell中上传文本

hadoop fs -put ./eg.txt /

3、进入share/hadoop/mapreduce目录下,启动排序

hadoop jar hadoop-mapreduce-examples-2.6..jar wordcount /eg.txt /out

4、屏幕输出结果如下:

16/03/29 21:30:26 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
16/03/29 21:30:30 INFO input.FileInputFormat: Total input paths to process : 1
16/03/29 21:30:30 INFO mapreduce.JobSubmitter: number of splits:1
16/03/29 21:30:31 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1459233715960_0004
16/03/29 21:30:31 INFO impl.YarnClientImpl: Submitted application application_1459233715960_0004
16/03/29 21:30:31 INFO mapreduce.Job: The url to track the job: http://m1.fredlab.org:8088/proxy/application_1459233715960_0004/
16/03/29 21:30:31 INFO mapreduce.Job: Running job: job_1459233715960_0004
16/03/29 21:30:47 INFO mapreduce.Job: Job job_1459233715960_0004 running in uber mode : false
16/03/29 21:30:47 INFO mapreduce.Job: map 0% reduce 0%
16/03/29 21:30:57 INFO mapreduce.Job: map 100% reduce 0%
16/03/29 21:31:09 INFO mapreduce.Job: map 100% reduce 100%
16/03/29 21:31:10 INFO mapreduce.Job: Job job_1459233715960_0004 completed successfully
16/03/29 21:31:11 INFO mapreduce.Job: Counters: 49
File System Counters
FILE: Number of bytes read=547
FILE: Number of bytes written=213761
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=453
HDFS: Number of bytes written=361
HDFS: Number of read operations=6
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=1
Launched reduce tasks=1
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=7594
Total time spent by all reduces in occupied slots (ms)=9087
Total time spent by all map tasks (ms)=7594
Total time spent by all reduce tasks (ms)=9087
Total vcore-milliseconds taken by all map tasks=7594
Total vcore-milliseconds taken by all reduce tasks=9087
Total megabyte-milliseconds taken by all map tasks=7776256
Total megabyte-milliseconds taken by all reduce tasks=9305088
Map-Reduce Framework
Map input records=11
Map output records=62
Map output bytes=598
Map output materialized bytes=547
Input split bytes=98
Combine input records=62
Combine output records=45
Reduce input groups=45
Reduce shuffle bytes=547
Reduce input records=45
Reduce output records=45
Spilled Records=90
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=310
CPU time spent (ms)=2010
Physical memory (bytes) snapshot=273182720
Virtual memory (bytes) snapshot=4122341376
Total committed heap usage (bytes)=137498624
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=355
File Output Format Counters
Bytes Written=361

5、结果文件位于hadoop集群/out目录下,如果执行成功,则出现_SUCCESS标识文件,并将结果存放于part-r-00000文件中。

Author:	1
EBook 1
Gutenberg 2
Leo 2
License 1
Peace 1
Peace, 1
Project 2
The 1
This 1
Title: 1
Tolstoy 2
War 2
You 1
almost 1
and 3
anyone 1
anywhere 1
at 2
away 1
by 1
copy 1
cost 1
eBook 2
for 1
give 1
included 1
is 1
it 2
it, 1
may 1
no 2
of 3
online 1
or 2
re-use 1
restrictions 1
terms 1
the 3
this 1
under 1
use 1
whatsoever. 1
with 2
www.gutenberg.org 1

可以到http://www.gutenberg.org/上下载更多txt版书籍文本来练习。

最新文章

  1. tomcat 7 WARNING: A context path must either be an empty string or start with a '/' and do not end with a '/'. The path [/] does not meet these criteria and has been changed to []
  2. Java魔法堂:解读基于Type Erasure的泛型
  3. MSP430G2333下位机乘法运算需要注意的一个问题
  4. vs2010调试程序出现“Cannot find or open the PDB file”
  5. hdu2896病毒侵袭(ac自动机)
  6. 关于调整浏览器窗口JS
  7. javascript------>(此文转发)
  8. QWaitCondition(和Java的Notify机制非常相像)
  9. C++中delete 和delete[]的区别
  10. 【转】真正的Acmer
  11. imx6平台qt锯齿原因分析
  12. LINUX 笔记-文件隐藏属性
  13. 【Teradata SQL】创建数据库和表
  14. JavaScript日历控件开发
  15. .NET MVC+angular导入导出
  16. linux中使用ps -ef
  17. HCatalog 学习之路
  18. 学习笔记-AngularJs(四)
  19. ETL项目场景
  20. 【转】SQL语句统计每天、每月、每年的数据

热门文章

  1. Serv-u FTP服务器
  2. 李洪强iOS开发之-环信02.1_环信 SDK 2.x到3.0升级文档
  3. webkit中DOM 事件有多少
  4. Android Gson使用笔记
  5. Android用户界面UI组件--AdapterView及其子类(四) GridView
  6. BZOJ1345: [Baltic2007]序列问题Sequence
  7. EXP-00056: ORACLE error 6550 encountered报错;
  8. 推荐一款自己的软件作品[豆约翰博客备份专家],新浪博客,QQ空间,CSDN,cnblogs博客备份,导出CHM,PDF(转载)
  9. Python手动构造Cookie模拟登录后获取网站页面内容
  10. poj 1556 The Doors(线段相交,最短路)