You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Kevin Tse <ke...@gmail.com> on 2010/06/14 15:51:53 UTC
No KeyValueTextInputFormat in hadoop-0.20.2?
Hi,
I am upgrading my code from hadoop-0.19.2 to hadoop-0.20.2, during the
process I found that there was no KeyValueTextInputFormat class which exists
in hadoop-0.19.2. It's so strange that this version of hadoop does not come
with this commonly used InputFormat. I have taken a look at the
"SecondarySort.java" example code, it uses TextInputFormat and
StringTokenizer to split each line, it is ok but kinda awkward to me.
Do I have to implement a new InputFormat myself or there's
a KeyValueTextInputFormat that exists somewhere I didn't notice?
Thank you.
Kevin Tse
Re: No KeyValueTextInputFormat in hadoop-0.20.2?
Posted by Amareshwari Sri Ramadasu <am...@yahoo-inc.com>.
The new api KeyValueTextInputFormat is not available in branch 0.20. It is added in branch 0.21 through https://issues.apache.org/jira/browse/MAPREDUCE-655.
Thanks
Amareshwari
On 6/21/10 6:52 AM, "Kevin Tse" <ke...@gmail.com> wrote:
Is there anybody knowing about this, please?
On Mon, Jun 14, 2010 at 10:21 PM, Kevin Tse <ke...@gmail.com>wrote:
> Hi Ted,
> I mean the new API:
>
> org.apache.hadoop.mapreduce.Job.setInputFormatClass(org.apache.hadoop.mapreduce.InputFormat)
>
> "Job.setInputFormatClass()" only accepts
> "org.apache.hadoop.mapreduce.InputFormat"(of which there are several
> subclasses, while KeyValueTextInputFormat is not one of them) as its
> parameter.
>
> On Mon, Jun 14, 2010 at 10:03 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Have you checked
>> src/mapred/org/apache/hadoop/mapred/KeyValueTextInputFormat.java ?
>>
>> On Mon, Jun 14, 2010 at 6:51 AM, Kevin Tse <ke...@gmail.com>
>> wrote:
>>
>> > Hi,
>> > I am upgrading my code from hadoop-0.19.2 to hadoop-0.20.2, during the
>> > process I found that there was no KeyValueTextInputFormat class which
>> > exists
>> > in hadoop-0.19.2. It's so strange that this version of hadoop does not
>> come
>> > with this commonly used InputFormat. I have taken a look at the
>> > "SecondarySort.java" example code, it uses TextInputFormat and
>> > StringTokenizer to split each line, it is ok but kinda awkward to me.
>> >
>> > Do I have to implement a new InputFormat myself or there's
>> > a KeyValueTextInputFormat that exists somewhere I didn't notice?
>> >
>> > Thank you.
>> > Kevin Tse
>> >
>>
>
>
Re: No KeyValueTextInputFormat in hadoop-0.20.2?
Posted by Kevin Tse <ke...@gmail.com>.
Is there anybody knowing about this, please?
On Mon, Jun 14, 2010 at 10:21 PM, Kevin Tse <ke...@gmail.com>wrote:
> Hi Ted,
> I mean the new API:
>
> org.apache.hadoop.mapreduce.Job.setInputFormatClass(org.apache.hadoop.mapreduce.InputFormat)
>
> "Job.setInputFormatClass()" only accepts
> "org.apache.hadoop.mapreduce.InputFormat"(of which there are several
> subclasses, while KeyValueTextInputFormat is not one of them) as its
> parameter.
>
> On Mon, Jun 14, 2010 at 10:03 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Have you checked
>> src/mapred/org/apache/hadoop/mapred/KeyValueTextInputFormat.java ?
>>
>> On Mon, Jun 14, 2010 at 6:51 AM, Kevin Tse <ke...@gmail.com>
>> wrote:
>>
>> > Hi,
>> > I am upgrading my code from hadoop-0.19.2 to hadoop-0.20.2, during the
>> > process I found that there was no KeyValueTextInputFormat class which
>> > exists
>> > in hadoop-0.19.2. It's so strange that this version of hadoop does not
>> come
>> > with this commonly used InputFormat. I have taken a look at the
>> > "SecondarySort.java" example code, it uses TextInputFormat and
>> > StringTokenizer to split each line, it is ok but kinda awkward to me.
>> >
>> > Do I have to implement a new InputFormat myself or there's
>> > a KeyValueTextInputFormat that exists somewhere I didn't notice?
>> >
>> > Thank you.
>> > Kevin Tse
>> >
>>
>
>
Re: No KeyValueTextInputFormat in hadoop-0.20.2?
Posted by Kevin Tse <ke...@gmail.com>.
Hi Ted,
I mean the new API:
org.apache.hadoop.mapreduce.Job.setInputFormatClass(org.apache.hadoop.mapreduce.InputFormat)
"Job.setInputFormatClass()" only accepts
"org.apache.hadoop.mapreduce.InputFormat"(of which there are several
subclasses, while KeyValueTextInputFormat is not one of them) as its
parameter.
On Mon, Jun 14, 2010 at 10:03 PM, Ted Yu <yu...@gmail.com> wrote:
> Have you checked
> src/mapred/org/apache/hadoop/mapred/KeyValueTextInputFormat.java ?
>
> On Mon, Jun 14, 2010 at 6:51 AM, Kevin Tse <ke...@gmail.com>
> wrote:
>
> > Hi,
> > I am upgrading my code from hadoop-0.19.2 to hadoop-0.20.2, during the
> > process I found that there was no KeyValueTextInputFormat class which
> > exists
> > in hadoop-0.19.2. It's so strange that this version of hadoop does not
> come
> > with this commonly used InputFormat. I have taken a look at the
> > "SecondarySort.java" example code, it uses TextInputFormat and
> > StringTokenizer to split each line, it is ok but kinda awkward to me.
> >
> > Do I have to implement a new InputFormat myself or there's
> > a KeyValueTextInputFormat that exists somewhere I didn't notice?
> >
> > Thank you.
> > Kevin Tse
> >
>
Re: No KeyValueTextInputFormat in hadoop-0.20.2?
Posted by Ted Yu <yu...@gmail.com>.
Have you checked
src/mapred/org/apache/hadoop/mapred/KeyValueTextInputFormat.java ?
On Mon, Jun 14, 2010 at 6:51 AM, Kevin Tse <ke...@gmail.com> wrote:
> Hi,
> I am upgrading my code from hadoop-0.19.2 to hadoop-0.20.2, during the
> process I found that there was no KeyValueTextInputFormat class which
> exists
> in hadoop-0.19.2. It's so strange that this version of hadoop does not come
> with this commonly used InputFormat. I have taken a look at the
> "SecondarySort.java" example code, it uses TextInputFormat and
> StringTokenizer to split each line, it is ok but kinda awkward to me.
>
> Do I have to implement a new InputFormat myself or there's
> a KeyValueTextInputFormat that exists somewhere I didn't notice?
>
> Thank you.
> Kevin Tse
>