You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Kim Vogt <ki...@simplegeo.com> on 2010/06/15 06:35:16 UTC

running elephant-bird in eclipse & codec property

Hi peeps,

I'm trying to run elephant-bird code in eclipse, specifically (
http://github.com/kevinweil/elephant-bird/blob/master/examples/src/pig/json_word_count.pig),
but I'm not sure how to set the core-site.xml properties via eclipse.  I
tried adding them to VM args but am still getting the following error:

10/06/14 21:23:34 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
10/06/14 21:23:34 WARN mapred.JobClient: No job jar file set.  User classes
may not be found. See JobConf(Class) or JobConf#setJar(String).
10/06/14 21:23:34 INFO input.FileInputFormat: Total input paths to process :
2
10/06/14 21:23:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
10/06/14 21:23:34 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 916aeae88ceb6734a679ebf9b48a93bea4cd9a06]
10/06/14 21:23:34 INFO input.LzoInputFormat: Added LZO split for
file:/home/kim/code/data/jsonData/json.txt.lzo[start=0, length=100]
10/06/14 21:23:34 INFO mapred.JobClient: Running job: job_local_0001
10/06/14 21:23:34 INFO input.FileInputFormat: Total input paths to process :
2
10/06/14 21:23:34 INFO input.LzoInputFormat: Added LZO split for
file:/home/kim/code/data/jsonData/json.txt.lzo[start=0, length=100]
10/06/14 21:23:34 INFO mapred.MapTask: io.sort.mb = 100
10/06/14 21:23:34 INFO mapred.MapTask: data buffer = 79691776/99614720
10/06/14 21:23:34 INFO mapred.MapTask: record buffer = 262144/327680
10/06/14 21:23:34 WARN mapred.LocalJobRunner: job_local_0001
java.io.IOException: No codec for file
file:/home/kim/code/data/jsonData/json.txt.lzo not found, cannot run
    at
com.twitter.elephantbird.mapreduce.input.LzoRecordReader.initialize(LzoRecordReader.java:64)
    at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:418)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:582)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
    at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)
10/06/14 21:23:35 INFO mapred.JobClient:  map 0% reduce 0%
10/06/14 21:23:35 INFO mapred.JobClient: Job complete: job_local_0001
10/06/14 21:23:35 INFO mapred.JobClient: Counters: 0

Help appreciated :-)

Thanks!

-Kim