Client side issues with implementing new Collections support in 1.2

[Rick Shaw <wf...@gmail.com>]

I am in the process of implementing collections support in the JDBC Driver and have run in to a few issues. Specifically:

1. There does not seem to be a notion of a reflexive o.a.c.cql.jdbc.* class for the new collection classes in o.a.c.db.marshall. Is that intentional or was that to be "left to be implemented student", which for me is now. I am trying to make sure I understand to broader design so I do not go off half cocked.

2. In order to do a proper job on the client side of translating the intent of the collection declarations that are built on the server side, the server must send enough data back in the thrift (or native) stream to reconstitute it properly on the client side. Unless I am missing something (very possible) I think the underlying base types of the data need to be in the encoded (ByteBuffer) value in addition to the literal values (encoded as a JSON string payload). JSON does not have a rich enough vocabulary to imply all of the CQL types without this additional information.

I think it is as simple as appending this information adjacent to the JSON literal and letting the compose/decompose methods deal with the exact content issues.

I am happy to cut a JIRA and provide a patch to handle the issues but want to air out the issues before solving something that messes with the bigger picture of the overall design intent that I may be missing.

Re: Client side issues with implementing new Collections support in 1.2

[Rick Shaw <wf...@gmail.com>]

If the data type comes across in the "generic" part of the type name in the
metadata, that should be enough for the client side to do its job. (i.e.
ListType<timestamp> or MapType<int,boolean> )

I'm happy to change the way to access the compose/decompose fucnctions and
handle the JDBC specific stuff based on Type name and not have a Jdbc
specific set of methods. Just commit in trunk and we will get going.

On Mon, Aug 6, 2012 at 11:20 AM, Jonathan Ellis <jb...@gmail.com> wrote:

> On Mon, Aug 6, 2012 at 10:15 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> > That said, it should still be possible to decode correctly by using
> > the type information from CqlMetadata.value_types.
>
> Hmm, apparently we only send "set", not "set<int>".  Probably a bug,
> but maybe it will be simpler to just switch to a more type-aware
> encoding as Sylvain suggested.
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>

Re: Client side issues with implementing new Collections support in 1.2

[Jonathan Ellis <jb...@gmail.com>]

On Mon, Aug 6, 2012 at 10:15 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> That said, it should still be possible to decode correctly by using
> the type information from CqlMetadata.value_types.

Hmm, apparently we only send "set", not "set<int>".  Probably a bug,
but maybe it will be simpler to just switch to a more type-aware
encoding as Sylvain suggested.

-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Client side issues with implementing new Collections support in 1.2

[Jonathan Ellis <jb...@gmail.com>]

On Mon, Aug 6, 2012 at 4:49 AM, Sylvain Lebresne <sy...@datastax.com> wrote:
>> 2. JSON does not have a rich enough vocabulary to imply all of the CQL types without this additional information.
>
> JSON really just was a kind of first try, because it made it easy to
> test the collection code. CASSANDRA-4453 brings a more suitable
> encoding (I note that the currently patch attached stick to json for
> thrift, but as I said in the comments, it's a one line change to
> modify that).

That said, it should still be possible to decode correctly by using
the type information from CqlMetadata.value_types.

-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Client side issues with implementing new Collections support in 1.2

[Sylvain Lebresne <sy...@datastax.com>]

> 1. There does not seem to be a notion of a reflexive o.a.c.cql.jdbc.* class for the new collection classes in o.a.c.db.marshall. Is that intentional or was that to be "left to be implemented student", which for me is now. I am trying to make sure I understand to broader design so I do not go off half cocked.

There isn't one yet, no. We probably need to get CASSANDRA-4453 in
first though, as it brings real compose and decompose methods for the
collections.

As a side note, I don't like at all that we have those jdbc type
classes inside Cassandra. That is, I understand that we want to allow
java clients to use the AbstractType classe without pulling all of C*,
but making that jdbc specific feels like a mistake. Anyway, that a
separate debate and I'll open a ticket for that, just wanted to bring
it up.

> 2. JSON does not have a rich enough vocabulary to imply all of the CQL types without this additional information.

JSON really just was a kind of first try, because it made it easy to
test the collection code. CASSANDRA-4453 brings a more suitable
encoding (I note that the currently patch attached stick to json for
thrift, but as I said in the comments, it's a one line change to
modify that).

--
Sylvain