You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Harsh J <ha...@cloudera.com> on 2012/11/24 12:02:11 UTC
Re: Argumments to Hadoop Command
Can you please share your code over pastebin.com or GH gists so?
Also, the mapreduce-dev@ address is for MR project development, not
user help. I've moved your question to the proper users list. Please
use that going forward :)
On Sat, Nov 24, 2012 at 11:45 AM, L N <nk...@gmail.com> wrote:
> Hi,
>
> I am trying to run a hadoop command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input output
>
> where input and output are directories.But I get the following error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://.com/user/krishna/input already exists
>
> If you see it is taking input directory as output and throwing output
> directory already exists.
>
> if I give below command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>
> I get below error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.InvalidInputException: Input path does not
> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>
> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
> input directory.
>
> Where am I doing wrong.
> I have implemented tool interface in main class.
>
>
> Thanks
> Lakshmi
--
Harsh J
RE: Argumments to Hadoop Command
Posted by "Kartashov, Andy" <An...@mpac.ca>.
With the Error: " Output directory hdfs://.com/user/krishna/input already exists" simply delete the output directory perior to runnign the job.
$hadoop fs -rm "output/file"
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Saturday, November 24, 2012 6:02 AM
To: nklnarayana@gmail.com; <us...@hadoop.apache.org>
Subject: Re: Argumments to Hadoop Command
Can you please share your code over pastebin.com or GH gists so?
Also, the mapreduce-dev@ address is for MR project development, not user help. I've moved your question to the proper users list. Please use that going forward :)
On Sat, Nov 24, 2012 at 11:45 AM, L N <nk...@gmail.com> wrote:
> Hi,
>
> I am trying to run a hadoop command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
> output
>
> where input and output are directories.But I get the following error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://.com/user/krishna/input already exists
>
> If you see it is taking input directory as output and throwing output
> directory already exists.
>
> if I give below command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>
> I get below error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.InvalidInputException: Input path does not
> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>
> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
> input directory.
>
> Where am I doing wrong.
> I have implemented tool interface in main class.
>
>
> Thanks
> Lakshmi
--
Harsh J
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
RE: Argumments to Hadoop Command
Posted by "Kartashov, Andy" <An...@mpac.ca>.
With the Error: " Output directory hdfs://.com/user/krishna/input already exists" simply delete the output directory perior to runnign the job.
$hadoop fs -rm "output/file"
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Saturday, November 24, 2012 6:02 AM
To: nklnarayana@gmail.com; <us...@hadoop.apache.org>
Subject: Re: Argumments to Hadoop Command
Can you please share your code over pastebin.com or GH gists so?
Also, the mapreduce-dev@ address is for MR project development, not user help. I've moved your question to the proper users list. Please use that going forward :)
On Sat, Nov 24, 2012 at 11:45 AM, L N <nk...@gmail.com> wrote:
> Hi,
>
> I am trying to run a hadoop command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
> output
>
> where input and output are directories.But I get the following error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://.com/user/krishna/input already exists
>
> If you see it is taking input directory as output and throwing output
> directory already exists.
>
> if I give below command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>
> I get below error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.InvalidInputException: Input path does not
> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>
> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
> input directory.
>
> Where am I doing wrong.
> I have implemented tool interface in main class.
>
>
> Thanks
> Lakshmi
--
Harsh J
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
RE: Argumments to Hadoop Command
Posted by "Kartashov, Andy" <An...@mpac.ca>.
With the Error: " Output directory hdfs://.com/user/krishna/input already exists" simply delete the output directory perior to runnign the job.
$hadoop fs -rm "output/file"
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Saturday, November 24, 2012 6:02 AM
To: nklnarayana@gmail.com; <us...@hadoop.apache.org>
Subject: Re: Argumments to Hadoop Command
Can you please share your code over pastebin.com or GH gists so?
Also, the mapreduce-dev@ address is for MR project development, not user help. I've moved your question to the proper users list. Please use that going forward :)
On Sat, Nov 24, 2012 at 11:45 AM, L N <nk...@gmail.com> wrote:
> Hi,
>
> I am trying to run a hadoop command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
> output
>
> where input and output are directories.But I get the following error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://.com/user/krishna/input already exists
>
> If you see it is taking input directory as output and throwing output
> directory already exists.
>
> if I give below command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>
> I get below error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.InvalidInputException: Input path does not
> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>
> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
> input directory.
>
> Where am I doing wrong.
> I have implemented tool interface in main class.
>
>
> Thanks
> Lakshmi
--
Harsh J
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
RE: Argumments to Hadoop Command
Posted by "Kartashov, Andy" <An...@mpac.ca>.
With the Error: " Output directory hdfs://.com/user/krishna/input already exists" simply delete the output directory perior to runnign the job.
$hadoop fs -rm "output/file"
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Saturday, November 24, 2012 6:02 AM
To: nklnarayana@gmail.com; <us...@hadoop.apache.org>
Subject: Re: Argumments to Hadoop Command
Can you please share your code over pastebin.com or GH gists so?
Also, the mapreduce-dev@ address is for MR project development, not user help. I've moved your question to the proper users list. Please use that going forward :)
On Sat, Nov 24, 2012 at 11:45 AM, L N <nk...@gmail.com> wrote:
> Hi,
>
> I am trying to run a hadoop command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
> output
>
> where input and output are directories.But I get the following error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://.com/user/krishna/input already exists
>
> If you see it is taking input directory as output and throwing output
> directory already exists.
>
> if I give below command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>
> I get below error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.InvalidInputException: Input path does not
> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>
> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
> input directory.
>
> Where am I doing wrong.
> I have implemented tool interface in main class.
>
>
> Thanks
> Lakshmi
--
Harsh J
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
Can you please share your "same error" again? There are two errors on
your original post - the latter should no longer occur at the very
least.
On Tue, Nov 27, 2012 at 2:17 AM, L N <nk...@gmail.com> wrote:
> Hi Harsh,
>
> Still I get the same error with your modified code.
>
> Hi Andy,
>
> It says output directory "input" does not exist. It is not properly taking
> the arguments
>
>
> Thanks
> Lakshmi
>
> On Mon, Nov 26, 2012 at 10:42 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> I remember responding with a proper answer earlier, but here it goes
>> again anyway:
>>
>> """
>> Hi,
>>
>> Please check http://pastebin.com/phFZgkrQ for a proper version. When
>> using Tool/Configured, reuse the already prepared configuration (i.e.
>> getConf()) rather than constructing a new one, otherwise all CLI
>> parsed options are lost.
>> """
>>
>> On Sun, Nov 25, 2012 at 11:30 PM, L N <nk...@gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > I am trying to run a hadoop command
>> >
>> > hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
>> > output
>> >
>> > where input and output are directories.But I get the following error.
>> >
>> > WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>> > arguments. Applications should implement Tool for the same.
>> > Exception in thread "main"
>> > org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
>> > hdfs://.com/user/krishna/input already exists
>> >
>> > If you see it is taking input directory as output and throwing output
>> > directory already exists.
>> >
>> > if I give below command
>> >
>> > hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>> >
>> > I get below error.
>> >
>> > WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>> > arguments. Applications should implement Tool for the same.
>> > Exception in thread "main"
>> > org.apache.hadoop.mapred.InvalidInputException: Input path does not
>> > exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>> >
>> > Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
>> > input directory.
>> >
>> > Where am I doing wrong.
>> > I have implemented tool interface in main class.
>> >
>> > I pasted the code in pastbin.com. Link is below.
>> >
>> > http://pastebin.com/eN1BXuWh
>> >
>> >
>> > Thanks
>> > Lakshmi
>> >
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
Can you please share your "same error" again? There are two errors on
your original post - the latter should no longer occur at the very
least.
On Tue, Nov 27, 2012 at 2:17 AM, L N <nk...@gmail.com> wrote:
> Hi Harsh,
>
> Still I get the same error with your modified code.
>
> Hi Andy,
>
> It says output directory "input" does not exist. It is not properly taking
> the arguments
>
>
> Thanks
> Lakshmi
>
> On Mon, Nov 26, 2012 at 10:42 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> I remember responding with a proper answer earlier, but here it goes
>> again anyway:
>>
>> """
>> Hi,
>>
>> Please check http://pastebin.com/phFZgkrQ for a proper version. When
>> using Tool/Configured, reuse the already prepared configuration (i.e.
>> getConf()) rather than constructing a new one, otherwise all CLI
>> parsed options are lost.
>> """
>>
>> On Sun, Nov 25, 2012 at 11:30 PM, L N <nk...@gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > I am trying to run a hadoop command
>> >
>> > hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
>> > output
>> >
>> > where input and output are directories.But I get the following error.
>> >
>> > WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>> > arguments. Applications should implement Tool for the same.
>> > Exception in thread "main"
>> > org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
>> > hdfs://.com/user/krishna/input already exists
>> >
>> > If you see it is taking input directory as output and throwing output
>> > directory already exists.
>> >
>> > if I give below command
>> >
>> > hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>> >
>> > I get below error.
>> >
>> > WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>> > arguments. Applications should implement Tool for the same.
>> > Exception in thread "main"
>> > org.apache.hadoop.mapred.InvalidInputException: Input path does not
>> > exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>> >
>> > Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
>> > input directory.
>> >
>> > Where am I doing wrong.
>> > I have implemented tool interface in main class.
>> >
>> > I pasted the code in pastbin.com. Link is below.
>> >
>> > http://pastebin.com/eN1BXuWh
>> >
>> >
>> > Thanks
>> > Lakshmi
>> >
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
Can you please share your "same error" again? There are two errors on
your original post - the latter should no longer occur at the very
least.
On Tue, Nov 27, 2012 at 2:17 AM, L N <nk...@gmail.com> wrote:
> Hi Harsh,
>
> Still I get the same error with your modified code.
>
> Hi Andy,
>
> It says output directory "input" does not exist. It is not properly taking
> the arguments
>
>
> Thanks
> Lakshmi
>
> On Mon, Nov 26, 2012 at 10:42 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> I remember responding with a proper answer earlier, but here it goes
>> again anyway:
>>
>> """
>> Hi,
>>
>> Please check http://pastebin.com/phFZgkrQ for a proper version. When
>> using Tool/Configured, reuse the already prepared configuration (i.e.
>> getConf()) rather than constructing a new one, otherwise all CLI
>> parsed options are lost.
>> """
>>
>> On Sun, Nov 25, 2012 at 11:30 PM, L N <nk...@gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > I am trying to run a hadoop command
>> >
>> > hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
>> > output
>> >
>> > where input and output are directories.But I get the following error.
>> >
>> > WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>> > arguments. Applications should implement Tool for the same.
>> > Exception in thread "main"
>> > org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
>> > hdfs://.com/user/krishna/input already exists
>> >
>> > If you see it is taking input directory as output and throwing output
>> > directory already exists.
>> >
>> > if I give below command
>> >
>> > hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>> >
>> > I get below error.
>> >
>> > WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>> > arguments. Applications should implement Tool for the same.
>> > Exception in thread "main"
>> > org.apache.hadoop.mapred.InvalidInputException: Input path does not
>> > exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>> >
>> > Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
>> > input directory.
>> >
>> > Where am I doing wrong.
>> > I have implemented tool interface in main class.
>> >
>> > I pasted the code in pastbin.com. Link is below.
>> >
>> > http://pastebin.com/eN1BXuWh
>> >
>> >
>> > Thanks
>> > Lakshmi
>> >
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
Can you please share your "same error" again? There are two errors on
your original post - the latter should no longer occur at the very
least.
On Tue, Nov 27, 2012 at 2:17 AM, L N <nk...@gmail.com> wrote:
> Hi Harsh,
>
> Still I get the same error with your modified code.
>
> Hi Andy,
>
> It says output directory "input" does not exist. It is not properly taking
> the arguments
>
>
> Thanks
> Lakshmi
>
> On Mon, Nov 26, 2012 at 10:42 AM, Harsh J <ha...@cloudera.com> wrote:
>>
>> I remember responding with a proper answer earlier, but here it goes
>> again anyway:
>>
>> """
>> Hi,
>>
>> Please check http://pastebin.com/phFZgkrQ for a proper version. When
>> using Tool/Configured, reuse the already prepared configuration (i.e.
>> getConf()) rather than constructing a new one, otherwise all CLI
>> parsed options are lost.
>> """
>>
>> On Sun, Nov 25, 2012 at 11:30 PM, L N <nk...@gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > I am trying to run a hadoop command
>> >
>> > hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
>> > output
>> >
>> > where input and output are directories.But I get the following error.
>> >
>> > WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>> > arguments. Applications should implement Tool for the same.
>> > Exception in thread "main"
>> > org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
>> > hdfs://.com/user/krishna/input already exists
>> >
>> > If you see it is taking input directory as output and throwing output
>> > directory already exists.
>> >
>> > if I give below command
>> >
>> > hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>> >
>> > I get below error.
>> >
>> > WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>> > arguments. Applications should implement Tool for the same.
>> > Exception in thread "main"
>> > org.apache.hadoop.mapred.InvalidInputException: Input path does not
>> > exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>> >
>> > Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
>> > input directory.
>> >
>> > Where am I doing wrong.
>> > I have implemented tool interface in main class.
>> >
>> > I pasted the code in pastbin.com. Link is below.
>> >
>> > http://pastebin.com/eN1BXuWh
>> >
>> >
>> > Thanks
>> > Lakshmi
>> >
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
I remember responding with a proper answer earlier, but here it goes
again anyway:
"""
Hi,
Please check http://pastebin.com/phFZgkrQ for a proper version. When
using Tool/Configured, reuse the already prepared configuration (i.e.
getConf()) rather than constructing a new one, otherwise all CLI
parsed options are lost.
"""
On Sun, Nov 25, 2012 at 11:30 PM, L N <nk...@gmail.com> wrote:
>
> Hi,
>
> I am trying to run a hadoop command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
> output
>
> where input and output are directories.But I get the following error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://.com/user/krishna/input already exists
>
> If you see it is taking input directory as output and throwing output
> directory already exists.
>
> if I give below command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>
> I get below error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.InvalidInputException: Input path does not
> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>
> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
> input directory.
>
> Where am I doing wrong.
> I have implemented tool interface in main class.
>
> I pasted the code in pastbin.com. Link is below.
>
> http://pastebin.com/eN1BXuWh
>
>
> Thanks
> Lakshmi
>
--
Harsh J
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
I remember responding with a proper answer earlier, but here it goes
again anyway:
"""
Hi,
Please check http://pastebin.com/phFZgkrQ for a proper version. When
using Tool/Configured, reuse the already prepared configuration (i.e.
getConf()) rather than constructing a new one, otherwise all CLI
parsed options are lost.
"""
On Sun, Nov 25, 2012 at 11:30 PM, L N <nk...@gmail.com> wrote:
>
> Hi,
>
> I am trying to run a hadoop command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
> output
>
> where input and output are directories.But I get the following error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://.com/user/krishna/input already exists
>
> If you see it is taking input directory as output and throwing output
> directory already exists.
>
> if I give below command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>
> I get below error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.InvalidInputException: Input path does not
> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>
> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
> input directory.
>
> Where am I doing wrong.
> I have implemented tool interface in main class.
>
> I pasted the code in pastbin.com. Link is below.
>
> http://pastebin.com/eN1BXuWh
>
>
> Thanks
> Lakshmi
>
--
Harsh J
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
I remember responding with a proper answer earlier, but here it goes
again anyway:
"""
Hi,
Please check http://pastebin.com/phFZgkrQ for a proper version. When
using Tool/Configured, reuse the already prepared configuration (i.e.
getConf()) rather than constructing a new one, otherwise all CLI
parsed options are lost.
"""
On Sun, Nov 25, 2012 at 11:30 PM, L N <nk...@gmail.com> wrote:
>
> Hi,
>
> I am trying to run a hadoop command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
> output
>
> where input and output are directories.But I get the following error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://.com/user/krishna/input already exists
>
> If you see it is taking input directory as output and throwing output
> directory already exists.
>
> if I give below command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>
> I get below error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.InvalidInputException: Input path does not
> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>
> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
> input directory.
>
> Where am I doing wrong.
> I have implemented tool interface in main class.
>
> I pasted the code in pastbin.com. Link is below.
>
> http://pastebin.com/eN1BXuWh
>
>
> Thanks
> Lakshmi
>
--
Harsh J
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
I remember responding with a proper answer earlier, but here it goes
again anyway:
"""
Hi,
Please check http://pastebin.com/phFZgkrQ for a proper version. When
using Tool/Configured, reuse the already prepared configuration (i.e.
getConf()) rather than constructing a new one, otherwise all CLI
parsed options are lost.
"""
On Sun, Nov 25, 2012 at 11:30 PM, L N <nk...@gmail.com> wrote:
>
> Hi,
>
> I am trying to run a hadoop command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
> output
>
> where input and output are directories.But I get the following error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://.com/user/krishna/input already exists
>
> If you see it is taking input directory as output and throwing output
> directory already exists.
>
> if I give below command
>
> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>
> I get below error.
>
> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
> arguments. Applications should implement Tool for the same.
> Exception in thread "main"
> org.apache.hadoop.mapred.InvalidInputException: Input path does not
> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>
> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
> input directory.
>
> Where am I doing wrong.
> I have implemented tool interface in main class.
>
> I pasted the code in pastbin.com. Link is below.
>
> http://pastebin.com/eN1BXuWh
>
>
> Thanks
> Lakshmi
>
--
Harsh J
Argumments to Hadoop Command
Posted by L N <nk...@gmail.com>.
Hi,
I am trying to run a hadoop command
hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
output
where input and output are directories.But I get the following error.
WARN mapred.JobClient: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
Exception in thread "main"
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
hdfs://.com/user/krishna/input already exists
If you see it is taking input directory as output and throwing output
directory already exists.
if I give below command
hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
I get below error.
WARN mapred.JobClient: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
Exception in thread "main"
org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
input directory.
Where am I doing wrong.
I have implemented tool interface in main class.
*I pasted the code in pastbin.com. Link is below.
http://pastebin.com/eN1BXuWh*
Thanks
Lakshmi
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
Hi,
Please check http://pastebin.com/phFZgkrQ for a proper version. When
using Tool/Configured, reuse the already prepared configuration (i.e.
getConf()) rather than constructing a new one, otherwise all CLI
parsed options are lost.
On Sun, Nov 25, 2012 at 12:43 AM, L N <nk...@gmail.com> wrote:
> Hi Harsh,
>
> I pasted the code in pastbin.com. Link is below.
>
> http://pastebin.com/eN1BXuWh
>
> To which DL , do I need to send these questions. Also, please let me
> know to which user list, my post is moved. I could not find it.
>
> Thanks
> Lakshmi
>
> On Sat, Nov 24, 2012 at 3:02 AM, Harsh J <ha...@cloudera.com> wrote:
>> Can you please share your code over pastebin.com or GH gists so?
>>
>> Also, the mapreduce-dev@ address is for MR project development, not
>> user help. I've moved your question to the proper users list. Please
>> use that going forward :)
>>
>> On Sat, Nov 24, 2012 at 11:45 AM, L N <nk...@gmail.com> wrote:
>>> Hi,
>>>
>>> I am trying to run a hadoop command
>>>
>>> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input output
>>>
>>> where input and output are directories.But I get the following error.
>>>
>>> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>>> arguments. Applications should implement Tool for the same.
>>> Exception in thread "main"
>>> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
>>> hdfs://.com/user/krishna/input already exists
>>>
>>> If you see it is taking input directory as output and throwing output
>>> directory already exists.
>>>
>>> if I give below command
>>>
>>> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>>>
>>> I get below error.
>>>
>>> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>>> arguments. Applications should implement Tool for the same.
>>> Exception in thread "main"
>>> org.apache.hadoop.mapred.InvalidInputException: Input path does not
>>> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>>>
>>> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
>>> input directory.
>>>
>>> Where am I doing wrong.
>>> I have implemented tool interface in main class.
>>>
>>>
>>> Thanks
>>> Lakshmi
>>
>>
>>
>> --
>> Harsh J
--
Harsh J
Argumments to Hadoop Command
Posted by L N <nk...@gmail.com>.
Hi,
I am trying to run a hadoop command
hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
output
where input and output are directories.But I get the following error.
WARN mapred.JobClient: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
Exception in thread "main"
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
hdfs://.com/user/krishna/input already exists
If you see it is taking input directory as output and throwing output
directory already exists.
if I give below command
hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
I get below error.
WARN mapred.JobClient: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
Exception in thread "main"
org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
input directory.
Where am I doing wrong.
I have implemented tool interface in main class.
*I pasted the code in pastbin.com. Link is below.
http://pastebin.com/eN1BXuWh*
Thanks
Lakshmi
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
Hi,
Please check http://pastebin.com/phFZgkrQ for a proper version. When
using Tool/Configured, reuse the already prepared configuration (i.e.
getConf()) rather than constructing a new one, otherwise all CLI
parsed options are lost.
On Sun, Nov 25, 2012 at 12:43 AM, L N <nk...@gmail.com> wrote:
> Hi Harsh,
>
> I pasted the code in pastbin.com. Link is below.
>
> http://pastebin.com/eN1BXuWh
>
> To which DL , do I need to send these questions. Also, please let me
> know to which user list, my post is moved. I could not find it.
>
> Thanks
> Lakshmi
>
> On Sat, Nov 24, 2012 at 3:02 AM, Harsh J <ha...@cloudera.com> wrote:
>> Can you please share your code over pastebin.com or GH gists so?
>>
>> Also, the mapreduce-dev@ address is for MR project development, not
>> user help. I've moved your question to the proper users list. Please
>> use that going forward :)
>>
>> On Sat, Nov 24, 2012 at 11:45 AM, L N <nk...@gmail.com> wrote:
>>> Hi,
>>>
>>> I am trying to run a hadoop command
>>>
>>> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input output
>>>
>>> where input and output are directories.But I get the following error.
>>>
>>> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>>> arguments. Applications should implement Tool for the same.
>>> Exception in thread "main"
>>> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
>>> hdfs://.com/user/krishna/input already exists
>>>
>>> If you see it is taking input directory as output and throwing output
>>> directory already exists.
>>>
>>> if I give below command
>>>
>>> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>>>
>>> I get below error.
>>>
>>> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>>> arguments. Applications should implement Tool for the same.
>>> Exception in thread "main"
>>> org.apache.hadoop.mapred.InvalidInputException: Input path does not
>>> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>>>
>>> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
>>> input directory.
>>>
>>> Where am I doing wrong.
>>> I have implemented tool interface in main class.
>>>
>>>
>>> Thanks
>>> Lakshmi
>>
>>
>>
>> --
>> Harsh J
--
Harsh J
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
Hi,
Please check http://pastebin.com/phFZgkrQ for a proper version. When
using Tool/Configured, reuse the already prepared configuration (i.e.
getConf()) rather than constructing a new one, otherwise all CLI
parsed options are lost.
On Sun, Nov 25, 2012 at 12:43 AM, L N <nk...@gmail.com> wrote:
> Hi Harsh,
>
> I pasted the code in pastbin.com. Link is below.
>
> http://pastebin.com/eN1BXuWh
>
> To which DL , do I need to send these questions. Also, please let me
> know to which user list, my post is moved. I could not find it.
>
> Thanks
> Lakshmi
>
> On Sat, Nov 24, 2012 at 3:02 AM, Harsh J <ha...@cloudera.com> wrote:
>> Can you please share your code over pastebin.com or GH gists so?
>>
>> Also, the mapreduce-dev@ address is for MR project development, not
>> user help. I've moved your question to the proper users list. Please
>> use that going forward :)
>>
>> On Sat, Nov 24, 2012 at 11:45 AM, L N <nk...@gmail.com> wrote:
>>> Hi,
>>>
>>> I am trying to run a hadoop command
>>>
>>> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input output
>>>
>>> where input and output are directories.But I get the following error.
>>>
>>> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>>> arguments. Applications should implement Tool for the same.
>>> Exception in thread "main"
>>> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
>>> hdfs://.com/user/krishna/input already exists
>>>
>>> If you see it is taking input directory as output and throwing output
>>> directory already exists.
>>>
>>> if I give below command
>>>
>>> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>>>
>>> I get below error.
>>>
>>> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>>> arguments. Applications should implement Tool for the same.
>>> Exception in thread "main"
>>> org.apache.hadoop.mapred.InvalidInputException: Input path does not
>>> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>>>
>>> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
>>> input directory.
>>>
>>> Where am I doing wrong.
>>> I have implemented tool interface in main class.
>>>
>>>
>>> Thanks
>>> Lakshmi
>>
>>
>>
>> --
>> Harsh J
--
Harsh J
Re: Argumments to Hadoop Command
Posted by Harsh J <ha...@cloudera.com>.
Hi,
Please check http://pastebin.com/phFZgkrQ for a proper version. When
using Tool/Configured, reuse the already prepared configuration (i.e.
getConf()) rather than constructing a new one, otherwise all CLI
parsed options are lost.
On Sun, Nov 25, 2012 at 12:43 AM, L N <nk...@gmail.com> wrote:
> Hi Harsh,
>
> I pasted the code in pastbin.com. Link is below.
>
> http://pastebin.com/eN1BXuWh
>
> To which DL , do I need to send these questions. Also, please let me
> know to which user list, my post is moved. I could not find it.
>
> Thanks
> Lakshmi
>
> On Sat, Nov 24, 2012 at 3:02 AM, Harsh J <ha...@cloudera.com> wrote:
>> Can you please share your code over pastebin.com or GH gists so?
>>
>> Also, the mapreduce-dev@ address is for MR project development, not
>> user help. I've moved your question to the proper users list. Please
>> use that going forward :)
>>
>> On Sat, Nov 24, 2012 at 11:45 AM, L N <nk...@gmail.com> wrote:
>>> Hi,
>>>
>>> I am trying to run a hadoop command
>>>
>>> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input output
>>>
>>> where input and output are directories.But I get the following error.
>>>
>>> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>>> arguments. Applications should implement Tool for the same.
>>> Exception in thread "main"
>>> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
>>> hdfs://.com/user/krishna/input already exists
>>>
>>> If you see it is taking input directory as output and throwing output
>>> directory already exists.
>>>
>>> if I give below command
>>>
>>> hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>>>
>>> I get below error.
>>>
>>> WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>>> arguments. Applications should implement Tool for the same.
>>> Exception in thread "main"
>>> org.apache.hadoop.mapred.InvalidInputException: Input path does not
>>> exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>>>
>>> Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
>>> input directory.
>>>
>>> Where am I doing wrong.
>>> I have implemented tool interface in main class.
>>>
>>>
>>> Thanks
>>> Lakshmi
>>
>>
>>
>> --
>> Harsh J
--
Harsh J
Argumments to Hadoop Command
Posted by L N <nk...@gmail.com>.
Hi,
I am trying to run a hadoop command
hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
output
where input and output are directories.But I get the following error.
WARN mapred.JobClient: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
Exception in thread "main"
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
hdfs://.com/user/krishna/input already exists
If you see it is taking input directory as output and throwing output
directory already exists.
if I give below command
hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
I get below error.
WARN mapred.JobClient: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
Exception in thread "main"
org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
input directory.
Where am I doing wrong.
I have implemented tool interface in main class.
*I pasted the code in pastbin.com. Link is below.
http://pastebin.com/eN1BXuWh*
Thanks
Lakshmi
Argumments to Hadoop Command
Posted by L N <nk...@gmail.com>.
Hi,
I am trying to run a hadoop command
hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
output
where input and output are directories.But I get the following error.
WARN mapred.JobClient: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
Exception in thread "main"
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
hdfs://.com/user/krishna/input already exists
If you see it is taking input directory as output and throwing output
directory already exists.
if I give below command
hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
I get below error.
WARN mapred.JobClient: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
Exception in thread "main"
org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
input directory.
Where am I doing wrong.
I have implemented tool interface in main class.
*I pasted the code in pastbin.com. Link is below.
http://pastebin.com/eN1BXuWh*
Thanks
Lakshmi