Error: Unexpected -D while processing...

[Gabriel Moraru <gm...@mpi-bremen.de>]

Hi.

I get a strange error in Hadoop 0.18.3 (Cloudera) while I am trying to run a streaming job.
The error is: "ERROR streaming.StreamJob: Unexpected -D while processing -input|-output|-mapper|-combiner|-reducer| etc etc etc"
If I remove all -D parameters the job is accepted by Hadoop. However, I still have a warning: "WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same."

Anybody knows what this means?
Thanks

Re: Error: Unexpected -D while processing...

[Matt Massie <ma...@cloudera.com>]

Gabriel-

Removing the -D is the right thing to do.  The warning re:
GenericOptionsParser can be safely ignored.

If you want more info about the warning to put your mind at ease, take a look at
src/mapred/org/apache/hadoop/mapred/JobClient.java (line 1550 for
context on the warning)
src/core/org/apache/hadoop/util/Tool.java (for the Tool interface)

-Matt

On Wed, Sep 30, 2009 at 8:29 AM, Gabriel Moraru <gm...@mpi-bremen.de> wrote:
> Hi.
>
> I get a strange error in Hadoop 0.18.3 (Cloudera) while I am trying to run a streaming job.
> The error is: "ERROR streaming.StreamJob: Unexpected -D while processing -input|-output|-mapper|-combiner|-reducer| etc etc etc"
> If I remove all -D parameters the job is accepted by Hadoop. However, I still have a warning: "WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same."
>
> Anybody knows what this means?
> Thanks
>
>