You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by John Tipper <jo...@hotmail.com> on 2022/06/13 11:54:59 UTC

How to implement unnest() as udtf using Python?

Hi all,

Flink doesn’t support the unnest() function, which takes an array and creates a row for each element in the array. I have column containing an array of map<str, str> and I’d like to implement my own unnest.

I try this as an initial do-nothing implementation:

@udtf(result_types=Datatypes.MAP(
    key_type=Datatypes.STRING(), value_type=Datatypes.STRING()))
def my_unnest(arr):
    return []

I get an error when Flink starts:

No match found for function signature my_unnest(<VARCHAR(2147483647), VARCHAR(2147483647)) MAP ARRAY>)

Is there something that I’m missing in my definition please?

Many thanks,

John

Re: How to implement unnest() as udtf using Python?

Posted by Dian Fu <di...@gmail.com>.

I second Martijn, UNNEST should be supported.

Besides, regarding the above exception, could you share an example which
could reproduce this issue?

Regards,
Dian

On Mon, Jun 13, 2022 at 8:21 PM Martijn Visser <ma...@apache.org>
wrote:

> Hi John,
>
> You're mentioning that Flink doesn't support UNNEST, but it does [1].
> Would this work for you?
>
> Best regards,
>
> Martijn
>
> [1]
> https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/dev/table/sql/queries/joins/#array-expansion
>
>
> Op ma 13 jun. 2022 om 13:55 schreef John Tipper <jo...@hotmail.com>:
>
>> Hi all,
>>
>> Flink doesn’t support the unnest() function, which takes an array and
>> creates a row for each element in the array. I have column containing an
>> array of map<str, str> and I’d like to implement my own unnest.
>>
>> I try this as an initial do-nothing implementation:
>>
>> @udtf(result_types=Datatypes.MAP(
>>     key_type=Datatypes.STRING(), value_type=Datatypes.STRING()))
>> def my_unnest(arr):
>>     return []
>>
>> I get an error when Flink starts:
>>
>> No match found for function signature my_unnest(<VARCHAR(2147483647),
>> VARCHAR(2147483647)) MAP ARRAY>)
>>
>> Is there something that I’m missing in my definition please?
>>
>> Many thanks,
>>
>> John
>>
>

Re: How to implement unnest() as udtf using Python?

Posted by Martijn Visser <ma...@apache.org>.

Hi John,

You're mentioning that Flink doesn't support UNNEST, but it does [1]. Would
this work for you?

Best regards,

Martijn

[1]
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/dev/table/sql/queries/joins/#array-expansion


Op ma 13 jun. 2022 om 13:55 schreef John Tipper <jo...@hotmail.com>:

> Hi all,
>
> Flink doesn’t support the unnest() function, which takes an array and
> creates a row for each element in the array. I have column containing an
> array of map<str, str> and I’d like to implement my own unnest.
>
> I try this as an initial do-nothing implementation:
>
> @udtf(result_types=Datatypes.MAP(
>     key_type=Datatypes.STRING(), value_type=Datatypes.STRING()))
> def my_unnest(arr):
>     return []
>
> I get an error when Flink starts:
>
> No match found for function signature my_unnest(<VARCHAR(2147483647),
> VARCHAR(2147483647)) MAP ARRAY>)
>
> Is there something that I’m missing in my definition please?
>
> Many thanks,
>
> John
>