You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Jeremy Lewi <je...@lewi.us> on 2011/04/08 06:35:31 UTC
Can't set stream.addenvironment in job configuration file
Hi,
I'm trying to submit a streaming job using the -conf option to specify
a job configuration file.
One of the options in my configuration file is stream.addenvironment
but this option doesn't appear to be getting set; i.e when I run the
job and use the web interface to look at job.xml this property is
missing.
Does anybody have any ideas?
My jobconf file is:
<configuration>
<property><name>stream.addenvironment</name><value>"dumbo_mrbase_class=dumbo.backends.common.MapRedBase dumbo_jk_class=dumbo.backends.common.JoinKey dumbo_runinfo_class=dumbo.backends.streaming.StreamingRunInfo
PYTHONPATH=common.pyc:typedbytes-0.3.6-py2.6.egg"</value></property>
<property><name>stream.map.input</name><value>typedbytes</value></property>
<property><name>stream.reduce.input</name><value>typedbytes</value></property>
<property><name>stream.map.output</name><value>typedbytes</value></property>
<property><name>stream.reduce.output</name><value>typedbytes</value></property>
<property><name>mapred.job.name</name><value>group_clusters.py
(1/1)</value></property>
</configuration>
Is it possible that the value I set for stream.addenvironment in the
job configuration file is overwritten by the call
jobConf._set("stream.addenvironment",addTaskEnvironment_);
in StreamJob.setJobConf()?
I'm using CDH3B.
Thanks
Jeremy