概述:
利用wordcount做hadoop性能测试,依据count的数据规模增长进行性能分析评测
版本:
bin/hadoop version
Hadoop 2.3.0-cdh5.0.0
测试步骤:
1.利用randomtextwriter生成指定规模的测试集合
2.执行wordcount:
nohup bin/hadoop jar share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.3.0-cdh5.0.0.jar wordcount /home/test/mrinput50 /home/test/mroutput50 > wc.log 2>&1 &
3.评测内容:
Total time spent by all maps in occupied slots (ms)=1504892 Total time spent by all reduces in occupied slots (ms)=84038 Total time spent by all map tasks (ms)=1504892 Total time spent by all reduce tasks (ms)=84038 GC time elapsed (ms)=17285 CPU time spent (ms)=1812107
相关推荐
使用hadoop实现WordCount详细实验报告,配有环境变量配置截图以及实验运行及结果详细过程描述与截图
hadoop-wordcount测试程序,jar包,单词统计的不二之选
该代码为hadoop的经典wordcount代码,java实现。代码里有详细注解,适合于入学者。
Hadoop 用mapreduce实现Wordcount实例,绝对能用
Hadoop开发WordCount源码程序详细讲解,每一行都带注释说明。
包含配置hadoop过程中遇到的一些问题的解决办法和成功运行wordcount实例的步骤
一套eclipse中的hadoop开发环境搭建教程,附带了eclipse中的hadoop的两个demo,一个是wordcount、一个是sort,大家一起学习进步!
这是一个wordcount的一个简单实例jar包,仅仅用来做测试。 map类:org.apache.hadoop.wordcount.WordCountMapReduce$WordCountMapper reduce类 org.apache.hadoop.wordcount.WordCountMapReduce$WordCountReducer
Hadoop集群-WordCount运行详解.pdf Hadoop,Hadoop集群,WordCount,pdf
<artifactId>wordcount <version>0.0.1-SNAPSHOT <packaging>jar <name>wordcount <url>http://maven.apache.org</url> <project.build.sourceEncoding>UTF-8 <groupId>org.apache.hadoop ...
ubuntu上面运行hadoop自带的wordcount实例
hadoop wordcount2.0 包含省略标点,忽略大小写等内容
关于Hadoop的WordCount实例代码,能够实现实现单词计数。
hadoop1.2.1修改WordCount并编译 在其中加入控制台输出
hadoop wordcount 打包部署
大数据实验报告Hadoop编程实现wordcount单词统计程序附源码.doc
hadoop wordcount 实验,成功运行 maven项目(全部源代码)
在linux环境下部署的Hadoop集群-WordCount运行详解。
基于Windows eclipse maven Hadoop 的WordCount源码。下载后需要自己编译jar包,详情请看解压后的README.md
hadoop入门例子wordcount