You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by shkob1 <sh...@gmail.com> on 2019/03/27 17:06:29 UTC
Calcite SQL Map to Pojo Map
Im trying to convert a SQL query that has a select map[..] into a pojo with
Map (using tableEnv.toRestractedStream )
It seems to fail when the field requestedTypeInfo is GenericTypeInfo with
GenericType<java.util.Map> while the field type itself is MapTypeInfo with
Map<String,String>
Exception in thread "main" org.apache.flink.table.api.TableException: Result
field does not match requested type. Requested: GenericType<java.util.Map>;
Actual: Map<String, String>
Any suggestion?
Shahar
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Calcite SQL Map to Pojo Map
Posted by Rong Rong <wa...@gmail.com>.
I think the proper solution should not be Types.GENERIC(Map.class) as you
will not be able to do any success processing with the return object.
For example, Map['k', 'v'].get('k') will not work.
I think there might be some problem like you suggested that they are
handled as GenericType instead of Pojo type, so it is not utilizing the
correct serializer.
It would be great if you can share the complete code that generates the
exception.
--
Rong
On Thu, Mar 28, 2019 at 1:56 PM shkob1 <sh...@gmail.com> wrote:
> Apparently the solution is to force map creating using UDF and to have the
> UDF return Types.GENERIC(Map.class)
> That makes them compatible and treated both as GenericType
>
> Thanks!
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>
Re: Calcite SQL Map to Pojo Map
Posted by shkob1 <sh...@gmail.com>.
Apparently the solution is to force map creating using UDF and to have the
UDF return Types.GENERIC(Map.class)
That makes them compatible and treated both as GenericType
Thanks!
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Calcite SQL Map to Pojo Map
Posted by Shahar Cizer Kobrinsky <sh...@gmail.com>.
Based on this discussion
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/HashMap-HashSet-Serialization-Issue-td10899.html
this seems by design that HashMap/Map are handled as GenericTypes .
However that doesn't work with the query result table schema which
generates a Map type.
On Thu, Mar 28, 2019 at 10:32 AM Shahar Cizer Kobrinsky <
shahar.kobrinsky@gmail.com> wrote:
> Hey Rong,
>
> I don't think this is about a UDF, i reproduce the same exception with a
> simple map['a','b'] where the Pojo has a Map<String,String> property
> btw for the UDF i'm already doing it (clazz is based on the specific map
> im creating):
>
> @Override
> public TypeInformation<?> getResultType(Class<?>[] signature) {
> return Types.MAP(Types.STRING, TypeInformation.of(clazz));
> }
>
> The table schema looks good but looking at the PojoTypeInfo fields the Map
> field is a GenericType<Map> - this causes the exception to be thrown on
> TableEnvironment.generateRowConverterFunction
>
>
> On Thu, Mar 28, 2019 at 8:56 AM Rong Rong <wa...@gmail.com> wrote:
>
>> If your conversion is done using a UDF you need to override the
>> getResultType method [1] to explicitly specify the key and value type
>> information. As generic erasure will not preseve the <String,String> part
>> of your code.
>>
>> Thanks,
>> Rong
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/table/udfs.html#scalar-functions
>>
>> On Wed, Mar 27, 2019 at 10:14 AM shkob1 <sh...@gmail.com>
>> wrote:
>>
>>> Im trying to convert a SQL query that has a select map[..] into a pojo
>>> with
>>> Map (using tableEnv.toRestractedStream )
>>> It seems to fail when the field requestedTypeInfo is GenericTypeInfo with
>>> GenericType<java.util.Map> while the field type itself is MapTypeInfo
>>> with
>>> Map<String,String>
>>>
>>>
>>> Exception in thread "main" org.apache.flink.table.api.TableException:
>>> Result
>>> field does not match requested type. Requested:
>>> GenericType<java.util.Map>;
>>> Actual: Map<String, String>
>>>
>>> Any suggestion?
>>> Shahar
>>>
>>>
>>>
>>> --
>>> Sent from:
>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>>>
>>
Re: Calcite SQL Map to Pojo Map
Posted by Shahar Cizer Kobrinsky <sh...@gmail.com>.
Hey Rong,
I don't think this is about a UDF, i reproduce the same exception with a
simple map['a','b'] where the Pojo has a Map<String,String> property
btw for the UDF i'm already doing it (clazz is based on the specific map im
creating):
@Override
public TypeInformation<?> getResultType(Class<?>[] signature) {
return Types.MAP(Types.STRING, TypeInformation.of(clazz));
}
The table schema looks good but looking at the PojoTypeInfo fields the Map
field is a GenericType<Map> - this causes the exception to be thrown on
TableEnvironment.generateRowConverterFunction
On Thu, Mar 28, 2019 at 8:56 AM Rong Rong <wa...@gmail.com> wrote:
> If your conversion is done using a UDF you need to override the
> getResultType method [1] to explicitly specify the key and value type
> information. As generic erasure will not preseve the <String,String> part
> of your code.
>
> Thanks,
> Rong
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/table/udfs.html#scalar-functions
>
> On Wed, Mar 27, 2019 at 10:14 AM shkob1 <sh...@gmail.com>
> wrote:
>
>> Im trying to convert a SQL query that has a select map[..] into a pojo
>> with
>> Map (using tableEnv.toRestractedStream )
>> It seems to fail when the field requestedTypeInfo is GenericTypeInfo with
>> GenericType<java.util.Map> while the field type itself is MapTypeInfo with
>> Map<String,String>
>>
>>
>> Exception in thread "main" org.apache.flink.table.api.TableException:
>> Result
>> field does not match requested type. Requested:
>> GenericType<java.util.Map>;
>> Actual: Map<String, String>
>>
>> Any suggestion?
>> Shahar
>>
>>
>>
>> --
>> Sent from:
>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>>
>
Re: Calcite SQL Map to Pojo Map
Posted by Rong Rong <wa...@gmail.com>.
If your conversion is done using a UDF you need to override the
getResultType method [1] to explicitly specify the key and value type
information. As generic erasure will not preseve the <String,String> part
of your code.
Thanks,
Rong
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/table/udfs.html#scalar-functions
On Wed, Mar 27, 2019 at 10:14 AM shkob1 <sh...@gmail.com> wrote:
> Im trying to convert a SQL query that has a select map[..] into a pojo with
> Map (using tableEnv.toRestractedStream )
> It seems to fail when the field requestedTypeInfo is GenericTypeInfo with
> GenericType<java.util.Map> while the field type itself is MapTypeInfo with
> Map<String,String>
>
>
> Exception in thread "main" org.apache.flink.table.api.TableException:
> Result
> field does not match requested type. Requested: GenericType<java.util.Map>;
> Actual: Map<String, String>
>
> Any suggestion?
> Shahar
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>