You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Pratyush Das <re...@gmail.com> on 2022/05/01 23:21:53 UTC

Running hadoop mapreduce examples with input format specifier

Hi,

I tried executing the Join.java example in the Hadoop Mapreduce Examples
jar using the following invocation - hadoop jar
hadoop-3.3.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar
join -inFormat TextInputFormat /examples-input/ /examples-output/

I keep getting an error - java.lang.ClassNotFoundException: TextInputFormat

This is the given invocation of the join example -
join [-r <reduces>] [-inFormat <input format class>] [-outFormat <output
format class>] [-outKey <output key class>] [-outValue <output value
class>] [-joinOp <inner|outer|override>] [input]* <input> <output>

What are the possible values of inFormat and outFormat? I have also tried
"TextInputFormat.class" which did not work.

Thanks,

-- 
Pratyush Das

Re: Running hadoop mapreduce examples with input format specifier

Posted by Ayush Saxena <ay...@gmail.com>.
By default the key class is BytesWritable.class, In case you are using some
different class you can provide it as an extra argument, something like -outKey
LongWritable.class along with your existing arguments

On Mon, 2 May 2022 at 06:02, Pratyush Das <re...@gmail.com> wrote:

> With the invocation - hadoop jar
> hadoop-3.3.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar
> join -inFormat org.apache.hadoop.mapreduce.lib.input.TextInputFormat
> /examples-input/ /examples-output/, I get the error - java.lang.Exception:
> java.io.IOException: wrong key class: org.apache.hadoop.io.LongWritable is
> not class org.apache.hadoop.io.BytesWritable
>
> So, it looks like the previous issue was solved, but something else might
> be happening? Is it necessary to manually specify a key class?
>
> On Sun, 1 May 2022 at 19:47, Ayush Saxena <ay...@gmail.com> wrote:
>
>> Hi Pratyush,
>> Can you try the class along with the package:
>>
>> *org.apache.hadoop.mapreduce.lib.input.TextInputFormat*
>>
>> It should work. All subclasses of InputFormat.java should work.
>>
>> -Ayush
>>
>>
>> On 02-May-2022, at 4:53 AM, Pratyush Das <re...@gmail.com> wrote:
>>
>> 
>> Hi,
>>
>> I tried executing the Join.java example in the Hadoop Mapreduce Examples
>> jar using the following invocation - hadoop jar
>> hadoop-3.3.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar
>> join -inFormat TextInputFormat /examples-input/ /examples-output/
>>
>> I keep getting an error - java.lang.ClassNotFoundException:
>> TextInputFormat
>>
>> This is the given invocation of the join example -
>> join [-r <reduces>] [-inFormat <input format class>] [-outFormat <output
>> format class>] [-outKey <output key class>] [-outValue <output value
>> class>] [-joinOp <inner|outer|override>] [input]* <input> <output>
>>
>> What are the possible values of inFormat and outFormat? I have also tried
>> "TextInputFormat.class" which did not work.
>>
>> Thanks,
>>
>> --
>> Pratyush Das
>>
>>
>
> --
> Pratyush Das
>

Re: Running hadoop mapreduce examples with input format specifier

Posted by Pratyush Das <re...@gmail.com>.
With the invocation - hadoop jar
hadoop-3.3.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar
join -inFormat org.apache.hadoop.mapreduce.lib.input.TextInputFormat
/examples-input/ /examples-output/, I get the error - java.lang.Exception:
java.io.IOException: wrong key class: org.apache.hadoop.io.LongWritable is
not class org.apache.hadoop.io.BytesWritable

So, it looks like the previous issue was solved, but something else might
be happening? Is it necessary to manually specify a key class?

On Sun, 1 May 2022 at 19:47, Ayush Saxena <ay...@gmail.com> wrote:

> Hi Pratyush,
> Can you try the class along with the package:
>
> *org.apache.hadoop.mapreduce.lib.input.TextInputFormat*
>
> It should work. All subclasses of InputFormat.java should work.
>
> -Ayush
>
>
> On 02-May-2022, at 4:53 AM, Pratyush Das <re...@gmail.com> wrote:
>
> 
> Hi,
>
> I tried executing the Join.java example in the Hadoop Mapreduce Examples
> jar using the following invocation - hadoop jar
> hadoop-3.3.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar
> join -inFormat TextInputFormat /examples-input/ /examples-output/
>
> I keep getting an error - java.lang.ClassNotFoundException:
> TextInputFormat
>
> This is the given invocation of the join example -
> join [-r <reduces>] [-inFormat <input format class>] [-outFormat <output
> format class>] [-outKey <output key class>] [-outValue <output value
> class>] [-joinOp <inner|outer|override>] [input]* <input> <output>
>
> What are the possible values of inFormat and outFormat? I have also tried
> "TextInputFormat.class" which did not work.
>
> Thanks,
>
> --
> Pratyush Das
>
>

-- 
Pratyush Das

Re: Running hadoop mapreduce examples with input format specifier

Posted by Ayush Saxena <ay...@gmail.com>.
Hi Pratyush,
Can you try the class along with the package:

org.apache.hadoop.mapreduce.lib.input.TextInputFormat

It should work. All subclasses of InputFormat.java should work.

-Ayush

> 
> On 02-May-2022, at 4:53 AM, Pratyush Das <re...@gmail.com> wrote:
> 
> 
> Hi,
> 
> I tried executing the Join.java example in the Hadoop Mapreduce Examples jar using the following invocation - hadoop jar hadoop-3.3.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar join -inFormat TextInputFormat /examples-input/ /examples-output/
> 
> I keep getting an error - java.lang.ClassNotFoundException: TextInputFormat
> 
> This is the given invocation of the join example - 
> join [-r <reduces>] [-inFormat <input format class>] [-outFormat <output format class>] [-outKey <output key class>] [-outValue <output value class>] [-joinOp <inner|outer|override>] [input]* <input> <output>
> 
> What are the possible values of inFormat and outFormat? I have also tried "TextInputFormat.class" which did not work.
> 
> Thanks,
> 
> -- 
> Pratyush Das