You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Stuti Awasthi <st...@hcl.com> on 2013/01/14 11:27:29 UTC
TupleWritable Format
Hi,
I wanted to write the my input file which is in Text format to TupleWritable but when I collect the output using "output.collect(<intWritable>, <TupleWritable>)" its giving me empty tuple.
private final static IntWritable one = new IntWritable(1);
public void map(LongWritable key, Text value, OutputCollector<IntWritable, TupleWritable> output, Reporter reporter)throws IOException {
IntWritable i = new IntWritable();
int k = Integer.parseInt(key.toString());
i.set(k);
TupleWritable tuple = new TupleWritable(new Writable[] { one });
output.collect(i, tuple);
}
Output:
<Key> []
<Key> []
How can I populate the Tuple with desired values ?
Regards
Stuti Awasthi
::DISCLAIMER::
----------------------------------------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses and other defects.
----------------------------------------------------------------------------------------------------------------------------------------------------
RE: TupleWritable Format
Posted by Stuti Awasthi <st...@hcl.com>.
Thanks Harsh,
Actually I wanted tuplewritable for the MatrixMultiplication Job in Mahout which takes (<intWritable>, <TupleWritable>) as input but now I have created input file with (<intWritable>, <VectorWritable>) which worked perfectly fine for me.
Thanks
Stuti
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Monday, January 14, 2013 5:48 PM
To: <us...@hadoop.apache.org>
Subject: Re: TupleWritable Format
As the doc on http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapred/join/TupleWritable.html states, I don't think you should be relying on the join package's TupleWritable directly and should instead either use your own tested implementation or an alternative, better serializer such as Apache Avro.
On Mon, Jan 14, 2013 at 3:57 PM, Stuti Awasthi <st...@hcl.com>> wrote:
Hi,
I wanted to write the my input file which is in Text format to TupleWritable but when I collect the output using "output.collect(<intWritable>, <TupleWritable>)" its giving me empty tuple.
private final static IntWritable one = new IntWritable(1);
public void map(LongWritable key, Text value, OutputCollector<IntWritable, TupleWritable> output, Reporter reporter)throws IOException {
IntWritable i = new IntWritable();
int k = Integer.parseInt(key.toString());
i.set(k);
TupleWritable tuple = new TupleWritable(new Writable[] { one });
output.collect(i, tuple);
}
Output:
<Key> []
<Key> []
How can I populate the Tuple with desired values ?
Regards
Stuti Awasthi
::DISCLAIMER::
----------------------------------------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses and other defects.
----------------------------------------------------------------------------------------------------------------------------------------------------
--
Harsh J
RE: TupleWritable Format
Posted by Stuti Awasthi <st...@hcl.com>.
Thanks Harsh,
Actually I wanted tuplewritable for the MatrixMultiplication Job in Mahout which takes (<intWritable>, <TupleWritable>) as input but now I have created input file with (<intWritable>, <VectorWritable>) which worked perfectly fine for me.
Thanks
Stuti
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Monday, January 14, 2013 5:48 PM
To: <us...@hadoop.apache.org>
Subject: Re: TupleWritable Format
As the doc on http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapred/join/TupleWritable.html states, I don't think you should be relying on the join package's TupleWritable directly and should instead either use your own tested implementation or an alternative, better serializer such as Apache Avro.
On Mon, Jan 14, 2013 at 3:57 PM, Stuti Awasthi <st...@hcl.com>> wrote:
Hi,
I wanted to write the my input file which is in Text format to TupleWritable but when I collect the output using "output.collect(<intWritable>, <TupleWritable>)" its giving me empty tuple.
private final static IntWritable one = new IntWritable(1);
public void map(LongWritable key, Text value, OutputCollector<IntWritable, TupleWritable> output, Reporter reporter)throws IOException {
IntWritable i = new IntWritable();
int k = Integer.parseInt(key.toString());
i.set(k);
TupleWritable tuple = new TupleWritable(new Writable[] { one });
output.collect(i, tuple);
}
Output:
<Key> []
<Key> []
How can I populate the Tuple with desired values ?
Regards
Stuti Awasthi
::DISCLAIMER::
----------------------------------------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses and other defects.
----------------------------------------------------------------------------------------------------------------------------------------------------
--
Harsh J
RE: TupleWritable Format
Posted by Stuti Awasthi <st...@hcl.com>.
Thanks Harsh,
Actually I wanted tuplewritable for the MatrixMultiplication Job in Mahout which takes (<intWritable>, <TupleWritable>) as input but now I have created input file with (<intWritable>, <VectorWritable>) which worked perfectly fine for me.
Thanks
Stuti
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Monday, January 14, 2013 5:48 PM
To: <us...@hadoop.apache.org>
Subject: Re: TupleWritable Format
As the doc on http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapred/join/TupleWritable.html states, I don't think you should be relying on the join package's TupleWritable directly and should instead either use your own tested implementation or an alternative, better serializer such as Apache Avro.
On Mon, Jan 14, 2013 at 3:57 PM, Stuti Awasthi <st...@hcl.com>> wrote:
Hi,
I wanted to write the my input file which is in Text format to TupleWritable but when I collect the output using "output.collect(<intWritable>, <TupleWritable>)" its giving me empty tuple.
private final static IntWritable one = new IntWritable(1);
public void map(LongWritable key, Text value, OutputCollector<IntWritable, TupleWritable> output, Reporter reporter)throws IOException {
IntWritable i = new IntWritable();
int k = Integer.parseInt(key.toString());
i.set(k);
TupleWritable tuple = new TupleWritable(new Writable[] { one });
output.collect(i, tuple);
}
Output:
<Key> []
<Key> []
How can I populate the Tuple with desired values ?
Regards
Stuti Awasthi
::DISCLAIMER::
----------------------------------------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses and other defects.
----------------------------------------------------------------------------------------------------------------------------------------------------
--
Harsh J
RE: TupleWritable Format
Posted by Stuti Awasthi <st...@hcl.com>.
Thanks Harsh,
Actually I wanted tuplewritable for the MatrixMultiplication Job in Mahout which takes (<intWritable>, <TupleWritable>) as input but now I have created input file with (<intWritable>, <VectorWritable>) which worked perfectly fine for me.
Thanks
Stuti
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Monday, January 14, 2013 5:48 PM
To: <us...@hadoop.apache.org>
Subject: Re: TupleWritable Format
As the doc on http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapred/join/TupleWritable.html states, I don't think you should be relying on the join package's TupleWritable directly and should instead either use your own tested implementation or an alternative, better serializer such as Apache Avro.
On Mon, Jan 14, 2013 at 3:57 PM, Stuti Awasthi <st...@hcl.com>> wrote:
Hi,
I wanted to write the my input file which is in Text format to TupleWritable but when I collect the output using "output.collect(<intWritable>, <TupleWritable>)" its giving me empty tuple.
private final static IntWritable one = new IntWritable(1);
public void map(LongWritable key, Text value, OutputCollector<IntWritable, TupleWritable> output, Reporter reporter)throws IOException {
IntWritable i = new IntWritable();
int k = Integer.parseInt(key.toString());
i.set(k);
TupleWritable tuple = new TupleWritable(new Writable[] { one });
output.collect(i, tuple);
}
Output:
<Key> []
<Key> []
How can I populate the Tuple with desired values ?
Regards
Stuti Awasthi
::DISCLAIMER::
----------------------------------------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses and other defects.
----------------------------------------------------------------------------------------------------------------------------------------------------
--
Harsh J
Re: TupleWritable Format
Posted by Harsh J <ha...@cloudera.com>.
As the doc on
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapred/join/TupleWritable.htmlstates,
I don't think you should be relying on the join package's
TupleWritable directly and should instead either use your own tested
implementation or an alternative, better serializer such as Apache Avro.
On Mon, Jan 14, 2013 at 3:57 PM, Stuti Awasthi <st...@hcl.com> wrote:
> Hi,****
>
> ** **
>
> I wanted to write the my input file which is in Text format to
> TupleWritable but when I collect the output using
> “output.collect(<intWritable>, <TupleWritable>)” its giving me empty tuple.
> ****
>
> ** **
>
> private final static IntWritable one = new IntWritable(1);****
>
> ** **
>
> public void map(LongWritable key, Text value,
> OutputCollector<IntWritable, TupleWritable> output, Reporter
> reporter)throws IOException {****
>
>
> IntWritable i = new IntWritable();****
>
> int k =
> Integer.parseInt(key.toString());****
>
> i.set(k);*
> ***
>
>
> TupleWritable tuple = new TupleWritable(new Writable[] { one });****
>
>
> output.collect(i, tuple);****
>
> ****
>
> }****
>
> ** **
>
> *Output:*
>
> <Key> []****
>
> <Key> []****
>
> ** **
>
> How can I populate the Tuple with desired values ?****
>
> ** **
>
> Regards****
>
> Stuti Awasthi****
>
>
>
> ::DISCLAIMER::
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only.
> E-mail transmission is not guaranteed to be secure or error-free as
> information could be intercepted, corrupted,
> lost, destroyed, arrive late or incomplete, or may contain viruses in
> transmission. The e mail and its contents
> (with or without referred errors) shall therefore not attach any liability
> on the originator or HCL or its affiliates.
> Views or opinions, if any, presented in this email are solely those of the
> author and may not necessarily reflect the
> views or opinions of HCL or its affiliates. Any form of reproduction,
> dissemination, copying, disclosure, modification,
> distribution and / or publication of this message without the prior
> written consent of authorized representative of
> HCL is strictly prohibited. If you have received this email in error
> please delete it and notify the sender immediately.
> Before opening any email and/or attachments, please check them for viruses
> and other defects.
>
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
--
Harsh J
Re: TupleWritable Format
Posted by Harsh J <ha...@cloudera.com>.
As the doc on
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapred/join/TupleWritable.htmlstates,
I don't think you should be relying on the join package's
TupleWritable directly and should instead either use your own tested
implementation or an alternative, better serializer such as Apache Avro.
On Mon, Jan 14, 2013 at 3:57 PM, Stuti Awasthi <st...@hcl.com> wrote:
> Hi,****
>
> ** **
>
> I wanted to write the my input file which is in Text format to
> TupleWritable but when I collect the output using
> “output.collect(<intWritable>, <TupleWritable>)” its giving me empty tuple.
> ****
>
> ** **
>
> private final static IntWritable one = new IntWritable(1);****
>
> ** **
>
> public void map(LongWritable key, Text value,
> OutputCollector<IntWritable, TupleWritable> output, Reporter
> reporter)throws IOException {****
>
>
> IntWritable i = new IntWritable();****
>
> int k =
> Integer.parseInt(key.toString());****
>
> i.set(k);*
> ***
>
>
> TupleWritable tuple = new TupleWritable(new Writable[] { one });****
>
>
> output.collect(i, tuple);****
>
> ****
>
> }****
>
> ** **
>
> *Output:*
>
> <Key> []****
>
> <Key> []****
>
> ** **
>
> How can I populate the Tuple with desired values ?****
>
> ** **
>
> Regards****
>
> Stuti Awasthi****
>
>
>
> ::DISCLAIMER::
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only.
> E-mail transmission is not guaranteed to be secure or error-free as
> information could be intercepted, corrupted,
> lost, destroyed, arrive late or incomplete, or may contain viruses in
> transmission. The e mail and its contents
> (with or without referred errors) shall therefore not attach any liability
> on the originator or HCL or its affiliates.
> Views or opinions, if any, presented in this email are solely those of the
> author and may not necessarily reflect the
> views or opinions of HCL or its affiliates. Any form of reproduction,
> dissemination, copying, disclosure, modification,
> distribution and / or publication of this message without the prior
> written consent of authorized representative of
> HCL is strictly prohibited. If you have received this email in error
> please delete it and notify the sender immediately.
> Before opening any email and/or attachments, please check them for viruses
> and other defects.
>
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
--
Harsh J
Re: TupleWritable Format
Posted by Harsh J <ha...@cloudera.com>.
As the doc on
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapred/join/TupleWritable.htmlstates,
I don't think you should be relying on the join package's
TupleWritable directly and should instead either use your own tested
implementation or an alternative, better serializer such as Apache Avro.
On Mon, Jan 14, 2013 at 3:57 PM, Stuti Awasthi <st...@hcl.com> wrote:
> Hi,****
>
> ** **
>
> I wanted to write the my input file which is in Text format to
> TupleWritable but when I collect the output using
> “output.collect(<intWritable>, <TupleWritable>)” its giving me empty tuple.
> ****
>
> ** **
>
> private final static IntWritable one = new IntWritable(1);****
>
> ** **
>
> public void map(LongWritable key, Text value,
> OutputCollector<IntWritable, TupleWritable> output, Reporter
> reporter)throws IOException {****
>
>
> IntWritable i = new IntWritable();****
>
> int k =
> Integer.parseInt(key.toString());****
>
> i.set(k);*
> ***
>
>
> TupleWritable tuple = new TupleWritable(new Writable[] { one });****
>
>
> output.collect(i, tuple);****
>
> ****
>
> }****
>
> ** **
>
> *Output:*
>
> <Key> []****
>
> <Key> []****
>
> ** **
>
> How can I populate the Tuple with desired values ?****
>
> ** **
>
> Regards****
>
> Stuti Awasthi****
>
>
>
> ::DISCLAIMER::
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only.
> E-mail transmission is not guaranteed to be secure or error-free as
> information could be intercepted, corrupted,
> lost, destroyed, arrive late or incomplete, or may contain viruses in
> transmission. The e mail and its contents
> (with or without referred errors) shall therefore not attach any liability
> on the originator or HCL or its affiliates.
> Views or opinions, if any, presented in this email are solely those of the
> author and may not necessarily reflect the
> views or opinions of HCL or its affiliates. Any form of reproduction,
> dissemination, copying, disclosure, modification,
> distribution and / or publication of this message without the prior
> written consent of authorized representative of
> HCL is strictly prohibited. If you have received this email in error
> please delete it and notify the sender immediately.
> Before opening any email and/or attachments, please check them for viruses
> and other defects.
>
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
--
Harsh J
Re: TupleWritable Format
Posted by Harsh J <ha...@cloudera.com>.
As the doc on
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapred/join/TupleWritable.htmlstates,
I don't think you should be relying on the join package's
TupleWritable directly and should instead either use your own tested
implementation or an alternative, better serializer such as Apache Avro.
On Mon, Jan 14, 2013 at 3:57 PM, Stuti Awasthi <st...@hcl.com> wrote:
> Hi,****
>
> ** **
>
> I wanted to write the my input file which is in Text format to
> TupleWritable but when I collect the output using
> “output.collect(<intWritable>, <TupleWritable>)” its giving me empty tuple.
> ****
>
> ** **
>
> private final static IntWritable one = new IntWritable(1);****
>
> ** **
>
> public void map(LongWritable key, Text value,
> OutputCollector<IntWritable, TupleWritable> output, Reporter
> reporter)throws IOException {****
>
>
> IntWritable i = new IntWritable();****
>
> int k =
> Integer.parseInt(key.toString());****
>
> i.set(k);*
> ***
>
>
> TupleWritable tuple = new TupleWritable(new Writable[] { one });****
>
>
> output.collect(i, tuple);****
>
> ****
>
> }****
>
> ** **
>
> *Output:*
>
> <Key> []****
>
> <Key> []****
>
> ** **
>
> How can I populate the Tuple with desired values ?****
>
> ** **
>
> Regards****
>
> Stuti Awasthi****
>
>
>
> ::DISCLAIMER::
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only.
> E-mail transmission is not guaranteed to be secure or error-free as
> information could be intercepted, corrupted,
> lost, destroyed, arrive late or incomplete, or may contain viruses in
> transmission. The e mail and its contents
> (with or without referred errors) shall therefore not attach any liability
> on the originator or HCL or its affiliates.
> Views or opinions, if any, presented in this email are solely those of the
> author and may not necessarily reflect the
> views or opinions of HCL or its affiliates. Any form of reproduction,
> dissemination, copying, disclosure, modification,
> distribution and / or publication of this message without the prior
> written consent of authorized representative of
> HCL is strictly prohibited. If you have received this email in error
> please delete it and notify the sender immediately.
> Before opening any email and/or attachments, please check them for viruses
> and other defects.
>
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
--
Harsh J