You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Kimmo Björnsson <ki...@kimmo.se> on 2009/11/19 15:28:34 UTC
Tuple as value in Map - does it work?
Hi all.
I'm trying to use a map with a tuple as the value. From the
documentation it looks like it would be possible. But I just can't get
it to work.
Look at this small example. When it tries to parse the value of the
map-entry it won't accept a tuple. Do I need to implement an own
loader??
grunt> cat mapwithtuple;
[words#(word1,word2),title#anicetitle]
grunt> x = load 'mapwithtuple' as (m:map[]);
grunt> dump x;
2009-11-19 15:21:04,562 [main] WARN
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger
- org.apache.pig.builtin.PigStorage: Unable to interpret value
[B@420f9c40 in field being converted to type map, caught
ParseException <Encountered " "(" "( "" at line 1, column 8.
Was expecting one of:
<SIGNEDINTEGER> ...
<LONGINTEGER> ...
<DOUBLENUMBER> ...
<FLOATNUMBER> ...
<STRING> ...
> field discarded
2009-11-19 15:21:04,563 [main] INFO
org.apache.pig.backend.local.executionengine.LocalPigLauncher -
Successfully stored result in: "file:/tmp/temp1153167469/tmp608425705"
2009-11-19 15:21:04,563 [main] INFO
org.apache.pig.backend.local.executionengine.LocalPigLauncher -
Records written : 1
2009-11-19 15:21:04,563 [main] INFO
org.apache.pig.backend.local.executionengine.LocalPigLauncher - Bytes
written : 0
2009-11-19 15:21:04,563 [main] INFO
org.apache.pig.backend.local.executionengine.LocalPigLauncher - 100%
complete!
2009-11-19 15:21:04,563 [main] INFO
org.apache.pig.backend.local.executionengine.LocalPigLauncher -
Success!!
()
grunt>
/ Kimmo
Re: Tuple as value in Map - does it work?
Posted by hc busy <hc...@gmail.com>.
I've been swamped with work recently. But I've updated the ticket with
two sub-tasks. One of the task is to fix comparator to solve the
problem that we couldn't sort on values of map. It seems to me that if
we didn't fix this, people using the BinStorage format will encounter
the same sorting problem.
There is a separate sub-task to implement TypedPigStorage that infers
type correctly. There have been several people who insisted that
PigStorage that does not resolve value type as it was originally
implemented is the correct thing for it to be doing, so I've moved
that task into a separate sub-task of pig-1016 so that it can be
resolved separately.
Sadly, I've been very busy, trying to cope with working and the
economy, so i haven't had a chance to continue the fix. Sorry.
I'll try to put some time into this during the weekend if people think
this is a useful feature...
--busy
p.s. btw, the patch as it stands will allow you to build a pig that
allows the values of maps to be tuples or even another map. And it
will sort correctly. So if you have a big project and need it
urgently, give it a try.
On Thu, Nov 19, 2009 at 6:59 AM, Jeff Zhang <zj...@gmail.com> wrote:
> Hi Kimmo,
>
> This is a bug of pig.
>
> Here's the JIRA item http://issues.apache.org/jira/browse/PIG-1016
>
>
> Jeff Zhang
>
>
> 2009/11/19 Kimmo Björnsson <ki...@kimmo.se>
>
>> Hi all.
>>
>> I'm trying to use a map with a tuple as the value. From the
>> documentation it looks like it would be possible. But I just can't get
>> it to work.
>>
>> Look at this small example. When it tries to parse the value of the
>> map-entry it won't accept a tuple. Do I need to implement an own
>> loader??
>>
>> grunt> cat mapwithtuple;
>> [words#(word1,word2),title#anicetitle]
>> grunt> x = load 'mapwithtuple' as (m:map[]);
>> grunt> dump x;
>> 2009-11-19 15:21:04,562 [main] WARN
>>
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger
>> - org.apache.pig.builtin.PigStorage: Unable to interpret value
>> [B@420f9c40 in field being converted to type map, caught
>> ParseException <Encountered " "(" "( "" at line 1, column 8.
>> Was expecting one of:
>> <SIGNEDINTEGER> ...
>> <LONGINTEGER> ...
>> <DOUBLENUMBER> ...
>> <FLOATNUMBER> ...
>> <STRING> ...
>> > field discarded
>> 2009-11-19 15:21:04,563 [main] INFO
>> org.apache.pig.backend.local.executionengine.LocalPigLauncher -
>> Successfully stored result in: "file:/tmp/temp1153167469/tmp608425705"
>> 2009-11-19 15:21:04,563 [main] INFO
>> org.apache.pig.backend.local.executionengine.LocalPigLauncher -
>> Records written : 1
>> 2009-11-19 15:21:04,563 [main] INFO
>> org.apache.pig.backend.local.executionengine.LocalPigLauncher - Bytes
>> written : 0
>> 2009-11-19 15:21:04,563 [main] INFO
>> org.apache.pig.backend.local.executionengine.LocalPigLauncher - 100%
>> complete!
>> 2009-11-19 15:21:04,563 [main] INFO
>> org.apache.pig.backend.local.executionengine.LocalPigLauncher -
>> Success!!
>> ()
>> grunt>
>>
>> / Kimmo
>>
>
Re: Tuple as value in Map - does it work?
Posted by Jeff Zhang <zj...@gmail.com>.
Hi Kimmo,
This is a bug of pig.
Here's the JIRA item http://issues.apache.org/jira/browse/PIG-1016
Jeff Zhang
2009/11/19 Kimmo Björnsson <ki...@kimmo.se>
> Hi all.
>
> I'm trying to use a map with a tuple as the value. From the
> documentation it looks like it would be possible. But I just can't get
> it to work.
>
> Look at this small example. When it tries to parse the value of the
> map-entry it won't accept a tuple. Do I need to implement an own
> loader??
>
> grunt> cat mapwithtuple;
> [words#(word1,word2),title#anicetitle]
> grunt> x = load 'mapwithtuple' as (m:map[]);
> grunt> dump x;
> 2009-11-19 15:21:04,562 [main] WARN
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger
> - org.apache.pig.builtin.PigStorage: Unable to interpret value
> [B@420f9c40 in field being converted to type map, caught
> ParseException <Encountered " "(" "( "" at line 1, column 8.
> Was expecting one of:
> <SIGNEDINTEGER> ...
> <LONGINTEGER> ...
> <DOUBLENUMBER> ...
> <FLOATNUMBER> ...
> <STRING> ...
> > field discarded
> 2009-11-19 15:21:04,563 [main] INFO
> org.apache.pig.backend.local.executionengine.LocalPigLauncher -
> Successfully stored result in: "file:/tmp/temp1153167469/tmp608425705"
> 2009-11-19 15:21:04,563 [main] INFO
> org.apache.pig.backend.local.executionengine.LocalPigLauncher -
> Records written : 1
> 2009-11-19 15:21:04,563 [main] INFO
> org.apache.pig.backend.local.executionengine.LocalPigLauncher - Bytes
> written : 0
> 2009-11-19 15:21:04,563 [main] INFO
> org.apache.pig.backend.local.executionengine.LocalPigLauncher - 100%
> complete!
> 2009-11-19 15:21:04,563 [main] INFO
> org.apache.pig.backend.local.executionengine.LocalPigLauncher -
> Success!!
> ()
> grunt>
>
> / Kimmo
>