jython udfs

[Stan Rosenberg <sr...@proclivitysystems.com>]

Hi,

I have three constant udfs in jython:

@outputSchema("m:map[bag{tuple()}]")
def dummy1():
    return {"key":[("value1", "value2")]}

@outputSchema("m:map[tuple()]")
def dummy2():
    return {"key":("value1", "value2")}

# doesn't work!
@outputSchema("m:map[bag{}]")
def dummy3():
    return {"key":["value1", "value2"]}


The first two are fine. The last one which returns a map whose values
are lists does not work.  (I am getting ClassCastException:
java.lang.String cannot be cast to org.apache.pig.data.Tuple).
Could somebody explain why?

Thanks,

stan

Re: jython udfs

[Jonathan Coveney <jc...@gmail.com>]

Bags have to contain tuples. It's how the type is defined.

2011/10/12 Stan Rosenberg <sr...@proclivitysystems.com>

> Hi,
>
> I have three constant udfs in jython:
>
> @outputSchema("m:map[bag{tuple()}]")
> def dummy1():
>    return {"key":[("value1", "value2")]}
>
> @outputSchema("m:map[tuple()]")
> def dummy2():
>    return {"key":("value1", "value2")}
>
> # doesn't work!
> @outputSchema("m:map[bag{}]")
> def dummy3():
>    return {"key":["value1", "value2"]}
>
>
> The first two are fine. The last one which returns a map whose values
> are lists does not work.  (I am getting ClassCastException:
> java.lang.String cannot be cast to org.apache.pig.data.Tuple).
> Could somebody explain why?
>
> Thanks,
>
> stan
>

Re: jython udfs

[Thejas Nair <th...@hortonworks.com>]

> Part of my question was whether a list of individuals (in python) has
> a corresponding type in pig.  I now see that it doesn't.  Instead,  it
> must be converted to a list of tuples.

There is a patch from Woody to make these translations easier. - 
https://issues.apache.org/jira/browse/PIG-1942
Once there is some consensus on the details, we should have it in. (I 
hope to spend some time in next few weeks to drive that.)

-Thejas

Re: jython udfs

[Stan Rosenberg <sr...@proclivitysystems.com>]

On Thu, Oct 13, 2011 at 11:52 AM, Norbert Burger
<no...@gmail.com> wrote:
> Also the output schema for dummy3() doesn't match what's being returned.
>  You're returning a list of strings, but the outputschema specifies a bag,
> which translates into a list of tuples (of something, eg. strings).
>

Part of my question was whether a list of individuals (in python) has
a corresponding type in pig.  I now see that it doesn't.  Instead,  it
must be converted to a list of tuples.

Re: jython udfs

[Norbert Burger <no...@gmail.com>]

Also the output schema for dummy3() doesn't match what's being returned.
 You're returning a list of strings, but the outputschema specifies a bag,
which translates into a list of tuples (of something, eg. strings).

Norbert

On Wed, Oct 12, 2011 at 7:29 PM, Stan Rosenberg <
srosenberg@proclivitysystems.com> wrote:

> Hi,
>
> I have three constant udfs in jython:
>
> @outputSchema("m:map[bag{tuple()}]")
> def dummy1():
>    return {"key":[("value1", "value2")]}
>
> @outputSchema("m:map[tuple()]")
> def dummy2():
>    return {"key":("value1", "value2")}
>
> # doesn't work!
> @outputSchema("m:map[bag{}]")
> def dummy3():
>    return {"key":["value1", "value2"]}
>
>
> The first two are fine. The last one which returns a map whose values
> are lists does not work.  (I am getting ClassCastException:
> java.lang.String cannot be cast to org.apache.pig.data.Tuple).
> Could somebody explain why?
>
> Thanks,
>
> stan
>