You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Dinçer Kavraal <dk...@gmail.com> on 2012/01/17 14:43:01 UTC

libjars problem

Hello,

I am having troubles distributing jars on child tasks. *libjars *option
doesn't seem to work. Such as:

# bin/mahout seq2sparse -i seq -o vector-bigrams -a
com.me.solr.analyzer.CustomTurkishAnalyzer
-chunk 200 -wt tfidf -s 1 -md 1 -x 90 -ng 2 -ml 700 -seq -nv
Running on hadoop, using HADOOP_HOME=/opt/hadoop-0.20.205.0
No HADOOP_CONF_DIR set, using /opt/hadoop-0.20.205.0/src/conf
Warning: $HADOOP_HOME is deprecated.

12/01/17 13:27:05 WARN driver.MahoutDriver: Unable to add class: -libjars
12/01/17 13:27:05 WARN driver.MahoutDriver: No -libjars.props found on
classpath, will use command-line arguments only
*Unknown program '-libjars' chosen.*


I checked the script and this one is provoked:
bin/hadoop jar /opt/mahout-distribution-0.5/mahout-examples-0.5-job.jar
org.apache.mahout.driver.MahoutDriver -libjars
:/opt/luceneLib/CustomTurkishAnalyzer.jar:/opt/luceneLib/lucene-analyzers-3.2.0.jar:/opt/luceneLib/lucene-benchmark-3.2.0.jar:/opt/luceneLib/lucene-core-3.2.0.jar:/opt/luceneLib/lucene-highlighter-3.2.0.jar:/opt/luceneLib/lucene-memory-3.2.0.jar:/opt/luceneLib/lucene-queries-3.2.0.jar:/opt/luceneLib/lucene-xercesImpl-3.2.0.jar:/opt/luceneLib/netmera_turkish_filter.jar:/opt/luceneLib/solr-commons-csv-3.2.0.jar:/opt/luceneLib/solr-core-3.2.0.jar:/opt/luceneLib/zemberek-cekirdek-2.1.1.jar:/opt/luceneLib/zemberek-tr-2.1.1.jar
-files /opt/luceneLib/stopwords-tr.txt seq2sparse -i seq -o vector-bigrams
-a com.me.solr.analyzer.CustomTurkishAnalyzer -chunk 200 -wt tfidf -s 1 -md
1 -x 90 -ng 2 -ml 700 -seq -nv

I have both tried to use the -libjars option after CLASSNAME and before jar
file. It doesn't seem to work either way. Have any ideas?

As a PS: I have tried to use "-classpath ...jar:...jar:...jar" property for
HADOOP_TASKTRACKER_OPTS in hadoop-env


Thanks,
Dincer