You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Alfonso Olias Sanz <al...@gmail.com> on 2008/03/28 11:49:36 UTC
Errors running a Map task + third party libraries + .properties
Hi there
I have a Map/Reduce task that loads a .properties file. In the
generated jar file I have a
conf directory with this file. So the pat look like
conf/myfile.properties.
Running the Map/Reduce task inside eclipse (using the eclipse plugin),
works fine. But when I use the command hadoop jar MyJar.jar
inputFolder outputFolder
I always get this exception:
task_200803271613_0028_m_000004_2:
gaia.cu1.tools.exception.GaiaConfigurationException: Cannot load
configuration from file conf/idt.properties
task_200803271613_0028_m_000004_2: at
gaia.cu1.tools.util.props.PropertyLoader.load(PropertyLoader.java:163)
task_200803271613_0028_m_000004_2: at
gaia.cu3.idt.infraimpl.Launcher.loadProperties(Launcher.java:95)
task_200803271613_0028_m_000004_2: at
gaia.cu3.idtmapreduce.IdtMapper.setUp(IdtMapper.java:195)
task_200803271613_0028_m_000004_2: at
gaia.cu3.idtmapreduce.IdtMapper.map(IdtMapper.java:228)
task_200803271613_0028_m_000004_2: at
gaia.cu3.idtmapreduce.IdtMapper.map(IdtMapper.java:1)
task_200803271613_0028_m_000004_2: at
org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
task_200803271613_0028_m_000004_2: at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:208)
task_200803271613_0028_m_000004_2: at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2071)
task_200803271613_0028_m_000004_2: Caused by:
org.apache.commons.configuration.ConfigurationException: Cannot locate
configuration source idt.properties
Any idea??? The jar file also contains third party jar files and I
included in the manifest the classpath to the relative lib folder, and
it is working. But not for the properties files.
Thanks
Re: Errors running a Map task + third party libraries + .properties
Posted by Peeyush Bishnoi <pe...@yahoo-inc.com>.
Hello ,
I think in your code you have to specify the DistributedCache option to
distribute your properties/configuration file to the remote machines so
that application should load properties/configuration file while your
task is executing. For more information and how to use DistributedCache
please look into the following URL :
http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#DistributedCache
Thanks
---
Peeyush
On Fri, 2008-03-28 at 11:49 +0100, Alfonso Olias Sanz wrote:
> Hi there
>
> I have a Map/Reduce task that loads a .properties file. In the
> generated jar file I have a
> conf directory with this file. So the pat look like
> conf/myfile.properties.
>
> Running the Map/Reduce task inside eclipse (using the eclipse plugin),
> works fine. But when I use the command hadoop jar MyJar.jar
> inputFolder outputFolder
>
> I always get this exception:
>
> task_200803271613_0028_m_000004_2:
> gaia.cu1.tools.exception.GaiaConfigurationException: Cannot load
> configuration from file conf/idt.properties
> task_200803271613_0028_m_000004_2: at
> gaia.cu1.tools.util.props.PropertyLoader.load(PropertyLoader.java:163)
> task_200803271613_0028_m_000004_2: at
> gaia.cu3.idt.infraimpl.Launcher.loadProperties(Launcher.java:95)
> task_200803271613_0028_m_000004_2: at
> gaia.cu3.idtmapreduce.IdtMapper.setUp(IdtMapper.java:195)
> task_200803271613_0028_m_000004_2: at
> gaia.cu3.idtmapreduce.IdtMapper.map(IdtMapper.java:228)
> task_200803271613_0028_m_000004_2: at
> gaia.cu3.idtmapreduce.IdtMapper.map(IdtMapper.java:1)
> task_200803271613_0028_m_000004_2: at
> org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> task_200803271613_0028_m_000004_2: at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:208)
> task_200803271613_0028_m_000004_2: at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2071)
> task_200803271613_0028_m_000004_2: Caused by:
> org.apache.commons.configuration.ConfigurationException: Cannot locate
> configuration source idt.properties
>
> Any idea??? The jar file also contains third party jar files and I
> included in the manifest the classpath to the relative lib folder, and
> it is working. But not for the properties files.
> Thanks