You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Dmitri Bourlatchkov <dm...@gmail.com> on 2020/10/30 18:57:06 UTC
Avatica's Meta.Frame appears to be unable to serialize float and char
column values
Hello,
I noticed that if the rows passed to org.apache.calcite.avatica.Meta.Frame
contain Float values, the following exception [1] occurs when the frame is
sent to the client with PROTOBUF serialization.
Would this be considered a valid use case?
I can only reproduce it in unit tests or with custom code using the Avatica
server. If Avatica is corrected to a JDBC backend, it appears to always
convert float values to Double and this failure mode does not manifest.
A similar class cast problem occurs with char values.
What do you think?
If you think it is worth fixing, I should be able to submit a pull request
quickly. I believe the fix is fairly small.
Thanks,
Dmitri.
[1] Exception snippet:
java.lang.ClassCastException: java.lang.Long cannot be cast to
java.lang.Float
at
org.apache.calcite.avatica.remote.TypedValue.writeToProtoWithType(TypedValue.java:600)
at org.apache.calcite.avatica.remote.TypedValue.toProto(TypedValue.java:805)
at org.apache.calcite.avatica.Meta$Frame.serializeScalar(Meta.java:991)
at org.apache.calcite.avatica.Meta$Frame.parseColumn(Meta.java:977)
at org.apache.calcite.avatica.Meta$Frame.toProto(Meta.java:942)
at
org.apache.calcite.avatica.remote.Service$FetchResponse.serialize(Service.java:1468)
[...snip...]
Re: Avatica's Meta.Frame appears to be unable to serialize float and
char column values
Posted by Julian Hyde <jh...@apache.org>.
Thanks for the clarification. Yes, open a PR.
Please don't assume that it will be me that reviews the PR.
Julian
On Wed, Nov 4, 2020 at 7:14 AM Dmitri Bourlatchkov
<dm...@gmail.com> wrote:
>
> Hi Julian,
>
> Apologies for pinging directly. Does the information below answer your
> previous question about this or would you like me to go into more details?
>
> Does it sound worth a Jira ticket (and PR)?
>
> Thanks,
> Dmitri.
>
> On Fri, 30 Oct 2020 at 17:25, Dmitri Bourlatchkov <
> dmitri.bourlatchkov@gmail.com> wrote:
>
> > > [...] I wanted to get a better idea what circumstances cause it.
> >
> > In this particular case I came across this failure while experimenting
> > with SQL support in Stargate [1], which uses Apache Cassandra for storage
> > and Cassandra's float type is actually 32-bit [2], so it naturally maps to
> > java float.
> >
> > In my use case data from storage is wired to Avatica's Frame objects via a
> > custom Meta implementation.
> >
> > [1] https://github.com/stargate/stargate
> > [2] https://cassandra.apache.org/doc/latest/cql/types.html
> >
> > On Fri, 30 Oct 2020 at 14:57, Dmitri Bourlatchkov <
> > dmitri.bourlatchkov@gmail.com> wrote:
> >
> >> Hello,
> >>
> >> I noticed that if the rows passed to
> >> org.apache.calcite.avatica.Meta.Frame contain Float values, the following
> >> exception [1] occurs when the frame is sent to the client with PROTOBUF
> >> serialization.
> >>
> >> Would this be considered a valid use case?
> >>
> >> I can only reproduce it in unit tests or with custom code using the
> >> Avatica server. If Avatica is corrected to a JDBC backend, it appears to
> >> always convert float values to Double and this failure mode does not
> >> manifest.
> >>
> >> A similar class cast problem occurs with char values.
> >>
> >> What do you think?
> >>
> >> If you think it is worth fixing, I should be able to submit a pull
> >> request quickly. I believe the fix is fairly small.
> >>
> >> Thanks,
> >> Dmitri.
> >>
> >> [1] Exception snippet:
> >> java.lang.ClassCastException: java.lang.Long cannot be cast to
> >> java.lang.Float
> >> at
> >> org.apache.calcite.avatica.remote.TypedValue.writeToProtoWithType(TypedValue.java:600)
> >> at
> >> org.apache.calcite.avatica.remote.TypedValue.toProto(TypedValue.java:805)
> >> at org.apache.calcite.avatica.Meta$Frame.serializeScalar(Meta.java:991)
> >> at org.apache.calcite.avatica.Meta$Frame.parseColumn(Meta.java:977)
> >> at org.apache.calcite.avatica.Meta$Frame.toProto(Meta.java:942)
> >> at
> >> org.apache.calcite.avatica.remote.Service$FetchResponse.serialize(Service.java:1468)
> >> [...snip...]
> >>
> >
Re: Avatica's Meta.Frame appears to be unable to serialize float and
char column values
Posted by Dmitri Bourlatchkov <dm...@gmail.com>.
Hi Julian,
Apologies for pinging directly. Does the information below answer your
previous question about this or would you like me to go into more details?
Does it sound worth a Jira ticket (and PR)?
Thanks,
Dmitri.
On Fri, 30 Oct 2020 at 17:25, Dmitri Bourlatchkov <
dmitri.bourlatchkov@gmail.com> wrote:
> > [...] I wanted to get a better idea what circumstances cause it.
>
> In this particular case I came across this failure while experimenting
> with SQL support in Stargate [1], which uses Apache Cassandra for storage
> and Cassandra's float type is actually 32-bit [2], so it naturally maps to
> java float.
>
> In my use case data from storage is wired to Avatica's Frame objects via a
> custom Meta implementation.
>
> [1] https://github.com/stargate/stargate
> [2] https://cassandra.apache.org/doc/latest/cql/types.html
>
> On Fri, 30 Oct 2020 at 14:57, Dmitri Bourlatchkov <
> dmitri.bourlatchkov@gmail.com> wrote:
>
>> Hello,
>>
>> I noticed that if the rows passed to
>> org.apache.calcite.avatica.Meta.Frame contain Float values, the following
>> exception [1] occurs when the frame is sent to the client with PROTOBUF
>> serialization.
>>
>> Would this be considered a valid use case?
>>
>> I can only reproduce it in unit tests or with custom code using the
>> Avatica server. If Avatica is corrected to a JDBC backend, it appears to
>> always convert float values to Double and this failure mode does not
>> manifest.
>>
>> A similar class cast problem occurs with char values.
>>
>> What do you think?
>>
>> If you think it is worth fixing, I should be able to submit a pull
>> request quickly. I believe the fix is fairly small.
>>
>> Thanks,
>> Dmitri.
>>
>> [1] Exception snippet:
>> java.lang.ClassCastException: java.lang.Long cannot be cast to
>> java.lang.Float
>> at
>> org.apache.calcite.avatica.remote.TypedValue.writeToProtoWithType(TypedValue.java:600)
>> at
>> org.apache.calcite.avatica.remote.TypedValue.toProto(TypedValue.java:805)
>> at org.apache.calcite.avatica.Meta$Frame.serializeScalar(Meta.java:991)
>> at org.apache.calcite.avatica.Meta$Frame.parseColumn(Meta.java:977)
>> at org.apache.calcite.avatica.Meta$Frame.toProto(Meta.java:942)
>> at
>> org.apache.calcite.avatica.remote.Service$FetchResponse.serialize(Service.java:1468)
>> [...snip...]
>>
>
Re: Avatica's Meta.Frame appears to be unable to serialize float and
char column values
Posted by Dmitri Bourlatchkov <dm...@gmail.com>.
> [...] I wanted to get a better idea what circumstances cause it.
In this particular case I came across this failure while experimenting with
SQL support in Stargate [1], which uses Apache Cassandra for storage and
Cassandra's float type is actually 32-bit [2], so it naturally maps to java
float.
In my use case data from storage is wired to Avatica's Frame objects via a
custom Meta implementation.
[1] https://github.com/stargate/stargate
[2] https://cassandra.apache.org/doc/latest/cql/types.html
On Fri, 30 Oct 2020 at 14:57, Dmitri Bourlatchkov <
dmitri.bourlatchkov@gmail.com> wrote:
> Hello,
>
> I noticed that if the rows passed to org.apache.calcite.avatica.Meta.Frame
> contain Float values, the following exception [1] occurs when the frame is
> sent to the client with PROTOBUF serialization.
>
> Would this be considered a valid use case?
>
> I can only reproduce it in unit tests or with custom code using the
> Avatica server. If Avatica is corrected to a JDBC backend, it appears to
> always convert float values to Double and this failure mode does not
> manifest.
>
> A similar class cast problem occurs with char values.
>
> What do you think?
>
> If you think it is worth fixing, I should be able to submit a pull request
> quickly. I believe the fix is fairly small.
>
> Thanks,
> Dmitri.
>
> [1] Exception snippet:
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.Float
> at
> org.apache.calcite.avatica.remote.TypedValue.writeToProtoWithType(TypedValue.java:600)
> at
> org.apache.calcite.avatica.remote.TypedValue.toProto(TypedValue.java:805)
> at org.apache.calcite.avatica.Meta$Frame.serializeScalar(Meta.java:991)
> at org.apache.calcite.avatica.Meta$Frame.parseColumn(Meta.java:977)
> at org.apache.calcite.avatica.Meta$Frame.toProto(Meta.java:942)
> at
> org.apache.calcite.avatica.remote.Service$FetchResponse.serialize(Service.java:1468)
> [...snip...]
>
Re: Avatica's Meta.Frame appears to be unable to serialize float and
char column values
Posted by Julian Hyde <jh...@apache.org>.
I guess this happened because, as you say, 'float' and 'char' never
happen in the JDBC world. The next question is: do they ever
legitimately occur in the SQL world?
To be clear, this sounds like this is a bug, and I think we should fix
it, but I wanted to get a better idea what circumstances cause it.
(I'm not sure it pertains here, but I'll remind you all that SQL's
FLOAT type is a 64 bit floating point and therefore more similar to
Java's 'double' than Java's 'float' type. SQL's 32 bit floating point
is called REAL.)
Julian
On Fri, Oct 30, 2020 at 12:15 PM Dmitri Bourlatchkov
<dm...@gmail.com> wrote:
>
> Hello,
>
> I noticed that if the rows passed to org.apache.calcite.avatica.Meta.Frame
> contain Float values, the following exception [1] occurs when the frame is
> sent to the client with PROTOBUF serialization.
>
> Would this be considered a valid use case?
>
> I can only reproduce it in unit tests or with custom code using the Avatica
> server. If Avatica is corrected to a JDBC backend, it appears to always
> convert float values to Double and this failure mode does not manifest.
>
> A similar class cast problem occurs with char values.
>
> What do you think?
>
> If you think it is worth fixing, I should be able to submit a pull request
> quickly. I believe the fix is fairly small.
>
> Thanks,
> Dmitri.
>
> [1] Exception snippet:
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.Float
> at
> org.apache.calcite.avatica.remote.TypedValue.writeToProtoWithType(TypedValue.java:600)
> at org.apache.calcite.avatica.remote.TypedValue.toProto(TypedValue.java:805)
> at org.apache.calcite.avatica.Meta$Frame.serializeScalar(Meta.java:991)
> at org.apache.calcite.avatica.Meta$Frame.parseColumn(Meta.java:977)
> at org.apache.calcite.avatica.Meta$Frame.toProto(Meta.java:942)
> at
> org.apache.calcite.avatica.remote.Service$FetchResponse.serialize(Service.java:1468)
> [...snip...]