So, my most recent problem was that my hadoop streaming job was failing, and the tracking url was listing this exception under "Failed/Killed
Task Attempts":Digging around in the logs directory I found a fuller explanation:java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 127
at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:311)
at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:132)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)My groovy script was failing to launch - presumably because groovy wasn't visible on the path. My hacked solution was to hardcode the path to groovy in my script:env: groovy: No such file or directory
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 127And then, voila! It worked!#!/usr/bin/env /Users/user/Applications/groovy-1.6-RC-2/bin/groovy
Agile Retroflection of the Day
-
Yves Hanoulle asks, “If you could change 1 thing today what would it be?” as
the first question in his Agile Retroflection of the Day project. Today
being ...
1 day ago

1 comments:
Now describe how that felt when Hadoop started working. The feeling of knowledge garnished via exception hell. For anyone who thought coffee was the fuel of development, you are dead wrong. The true motivating factor is the rush of scraping know-how out of nine hundred seventy three failed attempts at running a mis-typed method name. The "i" before "e" rule never stung so bad!
Post a Comment