You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Vasiliki Kalavri <va...@gmail.com> on 2014/11/08 19:04:54 UTC
Re: Type extraction with generic variables
Hello again,
I tried out your suggestion and made a variation of the mapVertices method,
which takes the class of the new value as an argument.
In order to extract the type from the class, I use
the TypeExtractor.getForClass() method.
Is this what you had in mind?
When testing, this seems to work fine for basic types and for a custom
custom class I tried, but I get an error when trying to change the value
type to a Tuple. Do I have to deal with Tuples separately or is there a
better way to extract the type?
Thanks!
Cheers,
V.
On 31 October 2014 01:16, Vasiliki Kalavri <va...@gmail.com>
wrote:
> Hi Timo,
>
> thank you for the explanation!
> I guess I will try implementing ResultTypeQueryable then :)
>
> Cheers,
> Vasia.
>
> On 30 October 2014 13:01, Timo Walther <fl...@twalthr.com> wrote:
>
>> Hi Vasiliki,
>>
>> your error is a very common problem we have together with types. The
>> problem is that Java does type erasure, which means that
>>
>> return vertices.map(new ApplyMapperToVertex<K, VV>(mapper));
>>
>> becomes
>>
>> return vertices.map(new ApplyMapperToVertex(mapper));
>>
>> Therefore we don't have the types. But since we have "implements
>> MapFunction<Tuple2<K, VV>, Tuple2<K, VV>>" the type extractor infers the
>> output through the input (which is always known).
>>
>> The return type must be provided manually e.g. by implementing the
>> interface ResultTypeQueryable or by the a new API operator (see my last
>> mailing list mail).
>>
>> Regards,
>> Timo
>>
>>
>>
>> On 30.10.2014 11:59, Vasiliki Kalavri wrote:
>>
>>> Hi all,
>>>
>>> one of the operations we want to implement for the flink graph api is
>>> mapVertices, i.e. applying a mapper to the vertices of the graph.
>>> The current implementation assumes that the vertex value type remains the
>>> same:
>>> https://github.com/project-flink/flink-graph/blob/master/
>>> src/main/java/flink/graphs/Graph.java
>>> .
>>> However, we would like to be able to change the vertex value type. When
>>> trying this out, we got this error:
>>>
>>> Type of TypeVariable 'NV' in 'class flink.graphs.Graph$
>>> ApplyMapperToVertex'
>>> could not be determined. This is most likely a type erasure problem. The
>>> type extraction currently supports types with generic variables only in
>>> cases where all variables in the return type can be deduced from the
>>> input
>>> type(s).
>>>
>>> How could we solve this and support changing the vertex value type?
>>> Thank you!
>>>
>>> Cheers,
>>> Vasia.
>>>
>>>
>>
>
Re: Type extraction with generic variables
Posted by Vasiliki Kalavri <va...@gmail.com>.
Thanks Stephan!
Indeed, this worked fine :)
Cheers,
V.
On 10 November 2014 12:26, Stephan Ewen <se...@apache.org> wrote:
> Hi Vasia!
>
> I think in your case, it would be even better to use "getMapReturnTypes(
> MapFunction<IN, OUT> mapInterface, TypeInformation<IN> inType)"
>
> That way you can pass the function that you wrap (the one that maps the
> vertex values) and get its produced type.
>
>
> But I think the example shows that we need much better tooling for types
> hidden in nested functions...
>
> Stephan
>
>
> On Mon, Nov 10, 2014 at 11:00 AM, Timo Walther <fl...@twalthr.com> wrote:
>
> > Hey,
> >
> > the TypeExtractor.getForClass() is only intended for primitive types, if
> > you want to extract all kinds of types you should use
> > TypeExtractor.createTypeInfo(Type t) (a class is also a type). However, I
> > think it would be better if your method takes TypeInformation as an
> > argument instead of a class, because otherwise the user has to extend
> tuple
> > in order to provide the tuple field types.
> >
> > Regards,
> > Timo
> >
> >
> > On 08.11.2014 19:04, Vasiliki Kalavri wrote:
> >
> >> Hello again,
> >>
> >> I tried out your suggestion and made a variation of the mapVertices
> >> method,
> >> which takes the class of the new value as an argument.
> >> In order to extract the type from the class, I use
> >> the TypeExtractor.getForClass() method.
> >> Is this what you had in mind?
> >> When testing, this seems to work fine for basic types and for a custom
> >> custom class I tried, but I get an error when trying to change the value
> >> type to a Tuple. Do I have to deal with Tuples separately or is there a
> >> better way to extract the type?
> >>
> >> Thanks!
> >>
> >> Cheers,
> >> V.
> >>
> >> On 31 October 2014 01:16, Vasiliki Kalavri <va...@gmail.com>
> >> wrote:
> >>
> >> Hi Timo,
> >>>
> >>> thank you for the explanation!
> >>> I guess I will try implementing ResultTypeQueryable then :)
> >>>
> >>> Cheers,
> >>> Vasia.
> >>>
> >>> On 30 October 2014 13:01, Timo Walther <fl...@twalthr.com> wrote:
> >>>
> >>> Hi Vasiliki,
> >>>>
> >>>> your error is a very common problem we have together with types. The
> >>>> problem is that Java does type erasure, which means that
> >>>>
> >>>> return vertices.map(new ApplyMapperToVertex<K, VV>(mapper));
> >>>>
> >>>> becomes
> >>>>
> >>>> return vertices.map(new ApplyMapperToVertex(mapper));
> >>>>
> >>>> Therefore we don't have the types. But since we have "implements
> >>>> MapFunction<Tuple2<K, VV>, Tuple2<K, VV>>" the type extractor infers
> the
> >>>> output through the input (which is always known).
> >>>>
> >>>> The return type must be provided manually e.g. by implementing the
> >>>> interface ResultTypeQueryable or by the a new API operator (see my
> last
> >>>> mailing list mail).
> >>>>
> >>>> Regards,
> >>>> Timo
> >>>>
> >>>>
> >>>>
> >>>> On 30.10.2014 11:59, Vasiliki Kalavri wrote:
> >>>>
> >>>> Hi all,
> >>>>>
> >>>>> one of the operations we want to implement for the flink graph api is
> >>>>> mapVertices, i.e. applying a mapper to the vertices of the graph.
> >>>>> The current implementation assumes that the vertex value type remains
> >>>>> the
> >>>>> same:
> >>>>> https://github.com/project-flink/flink-graph/blob/master/
> >>>>> src/main/java/flink/graphs/Graph.java
> >>>>> .
> >>>>> However, we would like to be able to change the vertex value type.
> When
> >>>>> trying this out, we got this error:
> >>>>>
> >>>>> Type of TypeVariable 'NV' in 'class flink.graphs.Graph$
> >>>>> ApplyMapperToVertex'
> >>>>> could not be determined. This is most likely a type erasure problem.
> >>>>> The
> >>>>> type extraction currently supports types with generic variables only
> in
> >>>>> cases where all variables in the return type can be deduced from the
> >>>>> input
> >>>>> type(s).
> >>>>>
> >>>>> How could we solve this and support changing the vertex value type?
> >>>>> Thank you!
> >>>>>
> >>>>> Cheers,
> >>>>> Vasia.
> >>>>>
> >>>>>
> >>>>>
> >
>
Re: Type extraction with generic variables
Posted by Stephan Ewen <se...@apache.org>.
Hi Vasia!
I think in your case, it would be even better to use "getMapReturnTypes(
MapFunction<IN, OUT> mapInterface, TypeInformation<IN> inType)"
That way you can pass the function that you wrap (the one that maps the
vertex values) and get its produced type.
But I think the example shows that we need much better tooling for types
hidden in nested functions...
Stephan
On Mon, Nov 10, 2014 at 11:00 AM, Timo Walther <fl...@twalthr.com> wrote:
> Hey,
>
> the TypeExtractor.getForClass() is only intended for primitive types, if
> you want to extract all kinds of types you should use
> TypeExtractor.createTypeInfo(Type t) (a class is also a type). However, I
> think it would be better if your method takes TypeInformation as an
> argument instead of a class, because otherwise the user has to extend tuple
> in order to provide the tuple field types.
>
> Regards,
> Timo
>
>
> On 08.11.2014 19:04, Vasiliki Kalavri wrote:
>
>> Hello again,
>>
>> I tried out your suggestion and made a variation of the mapVertices
>> method,
>> which takes the class of the new value as an argument.
>> In order to extract the type from the class, I use
>> the TypeExtractor.getForClass() method.
>> Is this what you had in mind?
>> When testing, this seems to work fine for basic types and for a custom
>> custom class I tried, but I get an error when trying to change the value
>> type to a Tuple. Do I have to deal with Tuples separately or is there a
>> better way to extract the type?
>>
>> Thanks!
>>
>> Cheers,
>> V.
>>
>> On 31 October 2014 01:16, Vasiliki Kalavri <va...@gmail.com>
>> wrote:
>>
>> Hi Timo,
>>>
>>> thank you for the explanation!
>>> I guess I will try implementing ResultTypeQueryable then :)
>>>
>>> Cheers,
>>> Vasia.
>>>
>>> On 30 October 2014 13:01, Timo Walther <fl...@twalthr.com> wrote:
>>>
>>> Hi Vasiliki,
>>>>
>>>> your error is a very common problem we have together with types. The
>>>> problem is that Java does type erasure, which means that
>>>>
>>>> return vertices.map(new ApplyMapperToVertex<K, VV>(mapper));
>>>>
>>>> becomes
>>>>
>>>> return vertices.map(new ApplyMapperToVertex(mapper));
>>>>
>>>> Therefore we don't have the types. But since we have "implements
>>>> MapFunction<Tuple2<K, VV>, Tuple2<K, VV>>" the type extractor infers the
>>>> output through the input (which is always known).
>>>>
>>>> The return type must be provided manually e.g. by implementing the
>>>> interface ResultTypeQueryable or by the a new API operator (see my last
>>>> mailing list mail).
>>>>
>>>> Regards,
>>>> Timo
>>>>
>>>>
>>>>
>>>> On 30.10.2014 11:59, Vasiliki Kalavri wrote:
>>>>
>>>> Hi all,
>>>>>
>>>>> one of the operations we want to implement for the flink graph api is
>>>>> mapVertices, i.e. applying a mapper to the vertices of the graph.
>>>>> The current implementation assumes that the vertex value type remains
>>>>> the
>>>>> same:
>>>>> https://github.com/project-flink/flink-graph/blob/master/
>>>>> src/main/java/flink/graphs/Graph.java
>>>>> .
>>>>> However, we would like to be able to change the vertex value type. When
>>>>> trying this out, we got this error:
>>>>>
>>>>> Type of TypeVariable 'NV' in 'class flink.graphs.Graph$
>>>>> ApplyMapperToVertex'
>>>>> could not be determined. This is most likely a type erasure problem.
>>>>> The
>>>>> type extraction currently supports types with generic variables only in
>>>>> cases where all variables in the return type can be deduced from the
>>>>> input
>>>>> type(s).
>>>>>
>>>>> How could we solve this and support changing the vertex value type?
>>>>> Thank you!
>>>>>
>>>>> Cheers,
>>>>> Vasia.
>>>>>
>>>>>
>>>>>
>
Re: Type extraction with generic variables
Posted by Timo Walther <fl...@twalthr.com>.
Hey,
the TypeExtractor.getForClass() is only intended for primitive types, if
you want to extract all kinds of types you should use
TypeExtractor.createTypeInfo(Type t) (a class is also a type). However,
I think it would be better if your method takes TypeInformation as an
argument instead of a class, because otherwise the user has to extend
tuple in order to provide the tuple field types.
Regards,
Timo
On 08.11.2014 19:04, Vasiliki Kalavri wrote:
> Hello again,
>
> I tried out your suggestion and made a variation of the mapVertices method,
> which takes the class of the new value as an argument.
> In order to extract the type from the class, I use
> the TypeExtractor.getForClass() method.
> Is this what you had in mind?
> When testing, this seems to work fine for basic types and for a custom
> custom class I tried, but I get an error when trying to change the value
> type to a Tuple. Do I have to deal with Tuples separately or is there a
> better way to extract the type?
>
> Thanks!
>
> Cheers,
> V.
>
> On 31 October 2014 01:16, Vasiliki Kalavri <va...@gmail.com>
> wrote:
>
>> Hi Timo,
>>
>> thank you for the explanation!
>> I guess I will try implementing ResultTypeQueryable then :)
>>
>> Cheers,
>> Vasia.
>>
>> On 30 October 2014 13:01, Timo Walther <fl...@twalthr.com> wrote:
>>
>>> Hi Vasiliki,
>>>
>>> your error is a very common problem we have together with types. The
>>> problem is that Java does type erasure, which means that
>>>
>>> return vertices.map(new ApplyMapperToVertex<K, VV>(mapper));
>>>
>>> becomes
>>>
>>> return vertices.map(new ApplyMapperToVertex(mapper));
>>>
>>> Therefore we don't have the types. But since we have "implements
>>> MapFunction<Tuple2<K, VV>, Tuple2<K, VV>>" the type extractor infers the
>>> output through the input (which is always known).
>>>
>>> The return type must be provided manually e.g. by implementing the
>>> interface ResultTypeQueryable or by the a new API operator (see my last
>>> mailing list mail).
>>>
>>> Regards,
>>> Timo
>>>
>>>
>>>
>>> On 30.10.2014 11:59, Vasiliki Kalavri wrote:
>>>
>>>> Hi all,
>>>>
>>>> one of the operations we want to implement for the flink graph api is
>>>> mapVertices, i.e. applying a mapper to the vertices of the graph.
>>>> The current implementation assumes that the vertex value type remains the
>>>> same:
>>>> https://github.com/project-flink/flink-graph/blob/master/
>>>> src/main/java/flink/graphs/Graph.java
>>>> .
>>>> However, we would like to be able to change the vertex value type. When
>>>> trying this out, we got this error:
>>>>
>>>> Type of TypeVariable 'NV' in 'class flink.graphs.Graph$
>>>> ApplyMapperToVertex'
>>>> could not be determined. This is most likely a type erasure problem. The
>>>> type extraction currently supports types with generic variables only in
>>>> cases where all variables in the return type can be deduced from the
>>>> input
>>>> type(s).
>>>>
>>>> How could we solve this and support changing the vertex value type?
>>>> Thank you!
>>>>
>>>> Cheers,
>>>> Vasia.
>>>>
>>>>