Scalding初探之一：基于Scala的Hadoop利器

把你从写繁琐的Map-reduce Job中解放出来，写分布式跟写本地程序没两样，Scala真真代表着先进生产力的方向啊。

好的，今天开始直播基于Scala的Scalding啦，循序渐进地看以下页面：

https://github.com/twitter/scalding#scalding

https://github.com/twitter/scalding/wiki/Getting-Started

https://github.com/willf/scalding_cookbook

看到scalding-cookbook的时候，可以开始尝试写比Word Count更酷的Scalding程序了

 import com.twitter.scalding._

 // input (tsv)  // 0 1 2 3 4 5 6  // 22 kinds of love nn2 io nn1  // 12 large green eyes jj jj nn2  //

 // output (tsv)  // 22 of kinds/nn2_love/nn1  // 12 green large/jj_eyes/nn2

 class contextCountJob(args : Args) extends Job(args) {    val inSchema = ('count, 'w1 ,'w2, 'w3, 'pos1, 'pos2, 'pos3)

   val outSchema = ('count, 'word, 'context)

   Tsv(args("input"),inSchema)      .mapTo(inSchema -> outSchema) {        parts : (String, String, String, String, String, String, String) => {          val (count, w1, w2, w3, pos1, pos2, pos3) = parts          val context = "%s/%s_%s/%s".format(w1,pos1,w3,pos3)   (count, w2, context)   }   }    .write(Tsv(args("output")))  }

比较糟糕的是Scala语言新潮到博客园插件都不支持。。。

http://docs.kiji.org/userguides/express/1.0.1/basic-scala-scalding/

http://sujitpal.blogspot.com/2012/08/scalding-for-impatient.html

https://github.com/sujitpal/hia-examples/tree/master/scala/scalding-impatient/src/main/scala/com/mycompany/impatient

https://github.com/twitter/scalding/wiki/Fields-based-API-Reference

https://github.com/twitter/scalding/wiki/Scalding-Sources

https://github.com/twitter/scalding/wiki/Field-rules

https://github.com/twitter/scalding/wiki/API-Reference

https://github.com/twitter/scalding/wiki

http://twitter.github.io/scalding/com/twitter/scalding/package.html

https://github.com/deanwampler/scalding-workshop

推荐Twitter公开课的PPT（此处应有翻墙）