You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Dhruv Kumar <dk...@ecs.umass.edu> on 2011/06/21 22:14:28 UTC
Deserializing a MapWritable entry set.
I want to extract the key-value pairs from a MapWritable, cast them into
Integer (key) and Double (value) types, and add them to another collection.
I'm attempting the following but this code is incorrect.
// initialDistributionStripe is a MapWritable<IntWritable, DoubleWritable>
// initialProbabilities is of type Vector which can have (Integer, Double)
entries in it
for (Map.Entry<Writable, Writable> entry : initialDistributionStripe.
entrySet()) {
initialProbabilities.set(entry.getKey(), entry.getValue());
}
Is there a convenient way to do this?
Re: Deserializing a MapWritable entry set.
Posted by Harsh J <ha...@cloudera.com>.
Dhruv,
If the <Writable, Writable> are IntWritable and DoubleWritable
underneath, simply cast them properly to those types after
get{Key,Value}() and then use the appropriate method to get the
underlying value (Simple .get() in most cases).
Is this what you're looking for?
On Wed, Jun 22, 2011 at 1:44 AM, Dhruv Kumar <dk...@ecs.umass.edu> wrote:
> I want to extract the key-value pairs from a MapWritable, cast them into
> Integer (key) and Double (value) types, and add them to another collection.
> I'm attempting the following but this code is incorrect.
>
> // initialDistributionStripe is a MapWritable<IntWritable, DoubleWritable>
> // initialProbabilities is of type Vector which can have (Integer, Double)
> entries in it
>
> for (Map.Entry<Writable, Writable> entry : initialDistributionStripe.
> entrySet()) {
> initialProbabilities.set(entry.getKey(), entry.getValue());
> }
>
>
> Is there a convenient way to do this?
>
--
Harsh J
Re: Deserializing a MapWritable entry set.
Posted by Harsh J <ha...@cloudera.com>.
Alberto,
Dhruv's pretty active on the Apache Mahout project. Best to track
things down there? Am sure it would interest you much too --
http://mahout.apache.org ;-)
On Wed, Jun 22, 2011 at 1:48 AM, Alberto Andreotti
<al...@gmail.com> wrote:
> Never worked with maps before, btw what are you trying to calculate?
>
>
> alberto.
>
> On 21 June 2011 17:14, Dhruv Kumar <dk...@ecs.umass.edu> wrote:
>
>> I want to extract the key-value pairs from a MapWritable, cast them into
>> Integer (key) and Double (value) types, and add them to another collection.
>> I'm attempting the following but this code is incorrect.
>>
>> // initialDistributionStripe is a MapWritable<IntWritable, DoubleWritable>
>> // initialProbabilities is of type Vector which can have (Integer, Double)
>> entries in it
>>
>> for (Map.Entry<Writable, Writable> entry : initialDistributionStripe.
>> entrySet()) {
>> initialProbabilities.set(entry.getKey(), entry.getValue());
>> }
>>
>>
>> Is there a convenient way to do this?
>>
>
>
>
> --
> José Pablo Alberto Andreotti.
> Tel: 54 351 4730292
> Móvil: 54351156526363.
> MSN: albertoandreotti@gmail.com
> Skype: andreottialberto
>
--
Harsh J
Re: Deserializing a MapWritable entry set.
Posted by Dhruv Kumar <dk...@ecs.umass.edu>.
On Tue, Jun 21, 2011 at 4:32 PM, Harsh J <ha...@cloudera.com> wrote:
> ((IntWritable) entry.getKey()).get(); and similar.
>
Perfect! Thanks Harsh.
>
> On Wed, Jun 22, 2011 at 2:00 AM, Dhruv Kumar <dk...@ecs.umass.edu> wrote:
> > The exact problem I'm facing is the following:
> >
> > entry.getKey() and entry.getValue() return Writable types. How do I
> extract
> > the buried Int and Double? In case of IntWritable and DoubleWritable
> return
> > types, I could have used entry.getKey().get() and entry.getValue.get()
> and
> > it would have been fine.
> >
> > On Tue, Jun 21, 2011 at 4:18 PM, Alberto Andreotti <
> > albertoandreotti@gmail.com> wrote:
> >
> >> Never worked with maps before, btw what are you trying to calculate?
> >>
> >>
> > There is no calculation in this loop, it is just a conversion from one
> type
> > (MapWritable) produced by the reducer(s) to another type (Vector) which
> can
> > be consumed by some legacy code for actual processing.
> >
> >
> >>
> >> alberto.
> >>
> >> On 21 June 2011 17:14, Dhruv Kumar <dk...@ecs.umass.edu> wrote:
> >>
> >> > I want to extract the key-value pairs from a MapWritable, cast them
> into
> >> > Integer (key) and Double (value) types, and add them to another
> >> collection.
> >> > I'm attempting the following but this code is incorrect.
> >> >
> >> > // initialDistributionStripe is a MapWritable<IntWritable,
> >> DoubleWritable>
> >> > // initialProbabilities is of type Vector which can have (Integer,
> >> Double)
> >> > entries in it
> >> >
> >> > for (Map.Entry<Writable, Writable> entry : initialDistributionStripe.
> >> > entrySet()) {
> >> > initialProbabilities.set(entry.getKey(), entry.getValue());
> >> > }
> >> >
> >> >
> >> > Is there a convenient way to do this?
> >> >
> >>
> >>
> >>
> >> --
> >> José Pablo Alberto Andreotti.
> >> Tel: 54 351 4730292
> >> Móvil: 54351156526363.
> >> MSN: albertoandreotti@gmail.com
> >> Skype: andreottialberto
> >>
> >
>
>
>
> --
> Harsh J
>
Re: Deserializing a MapWritable entry set.
Posted by Harsh J <ha...@cloudera.com>.
((IntWritable) entry.getKey()).get(); and similar.
On Wed, Jun 22, 2011 at 2:00 AM, Dhruv Kumar <dk...@ecs.umass.edu> wrote:
> The exact problem I'm facing is the following:
>
> entry.getKey() and entry.getValue() return Writable types. How do I extract
> the buried Int and Double? In case of IntWritable and DoubleWritable return
> types, I could have used entry.getKey().get() and entry.getValue.get() and
> it would have been fine.
>
> On Tue, Jun 21, 2011 at 4:18 PM, Alberto Andreotti <
> albertoandreotti@gmail.com> wrote:
>
>> Never worked with maps before, btw what are you trying to calculate?
>>
>>
> There is no calculation in this loop, it is just a conversion from one type
> (MapWritable) produced by the reducer(s) to another type (Vector) which can
> be consumed by some legacy code for actual processing.
>
>
>>
>> alberto.
>>
>> On 21 June 2011 17:14, Dhruv Kumar <dk...@ecs.umass.edu> wrote:
>>
>> > I want to extract the key-value pairs from a MapWritable, cast them into
>> > Integer (key) and Double (value) types, and add them to another
>> collection.
>> > I'm attempting the following but this code is incorrect.
>> >
>> > // initialDistributionStripe is a MapWritable<IntWritable,
>> DoubleWritable>
>> > // initialProbabilities is of type Vector which can have (Integer,
>> Double)
>> > entries in it
>> >
>> > for (Map.Entry<Writable, Writable> entry : initialDistributionStripe.
>> > entrySet()) {
>> > initialProbabilities.set(entry.getKey(), entry.getValue());
>> > }
>> >
>> >
>> > Is there a convenient way to do this?
>> >
>>
>>
>>
>> --
>> José Pablo Alberto Andreotti.
>> Tel: 54 351 4730292
>> Móvil: 54351156526363.
>> MSN: albertoandreotti@gmail.com
>> Skype: andreottialberto
>>
>
--
Harsh J
Re: Deserializing a MapWritable entry set.
Posted by Dhruv Kumar <dk...@ecs.umass.edu>.
The exact problem I'm facing is the following:
entry.getKey() and entry.getValue() return Writable types. How do I extract
the buried Int and Double? In case of IntWritable and DoubleWritable return
types, I could have used entry.getKey().get() and entry.getValue.get() and
it would have been fine.
On Tue, Jun 21, 2011 at 4:18 PM, Alberto Andreotti <
albertoandreotti@gmail.com> wrote:
> Never worked with maps before, btw what are you trying to calculate?
>
>
There is no calculation in this loop, it is just a conversion from one type
(MapWritable) produced by the reducer(s) to another type (Vector) which can
be consumed by some legacy code for actual processing.
>
> alberto.
>
> On 21 June 2011 17:14, Dhruv Kumar <dk...@ecs.umass.edu> wrote:
>
> > I want to extract the key-value pairs from a MapWritable, cast them into
> > Integer (key) and Double (value) types, and add them to another
> collection.
> > I'm attempting the following but this code is incorrect.
> >
> > // initialDistributionStripe is a MapWritable<IntWritable,
> DoubleWritable>
> > // initialProbabilities is of type Vector which can have (Integer,
> Double)
> > entries in it
> >
> > for (Map.Entry<Writable, Writable> entry : initialDistributionStripe.
> > entrySet()) {
> > initialProbabilities.set(entry.getKey(), entry.getValue());
> > }
> >
> >
> > Is there a convenient way to do this?
> >
>
>
>
> --
> José Pablo Alberto Andreotti.
> Tel: 54 351 4730292
> Móvil: 54351156526363.
> MSN: albertoandreotti@gmail.com
> Skype: andreottialberto
>
Re: Deserializing a MapWritable entry set.
Posted by Alberto Andreotti <al...@gmail.com>.
Never worked with maps before, btw what are you trying to calculate?
alberto.
On 21 June 2011 17:14, Dhruv Kumar <dk...@ecs.umass.edu> wrote:
> I want to extract the key-value pairs from a MapWritable, cast them into
> Integer (key) and Double (value) types, and add them to another collection.
> I'm attempting the following but this code is incorrect.
>
> // initialDistributionStripe is a MapWritable<IntWritable, DoubleWritable>
> // initialProbabilities is of type Vector which can have (Integer, Double)
> entries in it
>
> for (Map.Entry<Writable, Writable> entry : initialDistributionStripe.
> entrySet()) {
> initialProbabilities.set(entry.getKey(), entry.getValue());
> }
>
>
> Is there a convenient way to do this?
>
--
José Pablo Alberto Andreotti.
Tel: 54 351 4730292
Móvil: 54351156526363.
MSN: albertoandreotti@gmail.com
Skype: andreottialberto