You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Josh Elser <jo...@gmail.com> on 2015/08/11 19:16:33 UTC
Should Jackson-annotated POJOs stick around
Pulling this out to email to avoid cluttering JIRA. I feel like I might
not be on the same page.
I see CALCITE-839 and CALCITE-840 being one in the same change, or at
least the root cause being solved by it.
Julian, I'm getting the impression that you envision protocol buffer
encoding being just another option for encoding requests and responses.
My opinion is that using protocol buffers to define these requests and
responses completely invalidates the need to support these POJOs. These
objects should be usable cross-language, so aside from support the
releases of Calcite which shipped only these POJOs, I don't see a need
to maintain them.
I am admittedly hedging my bet that the PB devs will release a new
version that has the advertised JSON-esque serialization format (instead
of just a binary format). If this ultimately falls through, POJOs that
just wrap the PB classes could also be done.
I just wanted to make sure I'm not dancing by myself and that we're all
in agreement on a general direction.
-------- Original Message --------
Subject: [jira] [Commented] (CALCITE-839) Remove Jackson annotations
from POJO classes in Meta
Date: Tue, 11 Aug 2015 05:41:45 +0000 (UTC)
From: Julian Hyde (JIRA) <ji...@apache.org>
Reply-To: dev@calcite.incubator.apache.org
To: issues@calcite.incubator.apache.org
[
https://issues.apache.org/jira/browse/CALCITE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681279#comment-14681279
]
Julian Hyde commented on CALCITE-839:
-------------------------------------
Well then, I've assigned CALCITE-840 to you.
> Remove Jackson annotations from POJO classes in Meta
> ----------------------------------------------------
>
> Key: CALCITE-839
> URL: https://issues.apache.org/jira/browse/CALCITE-839
> Project: Calcite
> Issue Type: Bug
> Components: avatica
> Reporter: Julian Hyde
> Assignee: Julian Hyde
>
> The Meta interface contains several POJO classes that represent RPC requests or responses. Currently a few of those classes have Jackson annotations such as @JsonCreator, @JsonProperty to help Jackson serialize the POJO to JSON and de-serialize from JSON to the object.
> As [~ndimiduk] pointed out in http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201503.mbox/%3CCANZa=GvKGD+BKJ4+eJMuO6iVhS+OkGSkG1VwDAZCy-ZijyyyPw@mail.gmail.com%3E these annotations are a "code smell" and should be removed. It makes it look as if Jackson is the only possible transport, which is not the case. We can continue to use Jackson as a transport, just specify the mappings elsewhere, not as annotations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Should Jackson-annotated POJOs stick around
Posted by Paul Brown <pr...@mult.ifario.us>.
I can certainly second Ted's comment. I would go so far as to say that
Jackson's multi-encoding/protocol support is intended for exposing the same
POJOs multiple ways but should not be regarded as an endorsement of
coupling tiers of your application (internal serialization versus API/wire
serialization) together by reusing the same objects.
—
prb@mult.ifario.us | Multifarious, Inc. | http://mult.ifario.us/
On Tue, Aug 11, 2015 at 2:44 PM, Josh Elser <jo...@gmail.com> wrote:
> Ted Dunning wrote:
>
>> On Tue, Aug 11, 2015 at 1:14 PM, Josh Elser<jo...@gmail.com> wrote:
>>
>> Andrew Purtell wrote:
>>>
>>> That might be because protobuf documentation, and I'd assume accumulated
>>>> practice based upon it, warns against using generated pbuf objects
>>>> directly
>>>> as model classes. (See the "Protocol Buffers and O-O Design" callout on
>>>> https://developers.google.com/protocol-buffers/docs/javatutorial.)
>>>>
>>>> Assuming that's the case, that makes sense. It was just not clear to me
>>> if
>>> Julian and I were just talking past each other or if there was some
>>> fallacy
>>> I was suggesting.
>>>
>>
>>
>> This differentiation between wire protocol and API is something that I
>> have
>> seen repeatedly in ex-Googlers. I was a bit curious since it seemed nice
>> to
>> have one definition for both levels.
>>
>> My opinion has verged to be 100% with the Google philosophy of separation
>> after watching how the MapR internals work. This kind of separation has
>> really paid off in many instances. Having too tight a lock between wire
>> and
>> API would have been nearly disastrous for either comprehensibility of the
>> API or efficiency of the wire. I can't share specifics, but if second-hand
>> opinions are useful, you now have mine.
>>
>>
> Absolutely, opinions are very useful here, Ted. Much appreciated. I'm
> trying to feel my way through the cleanest approach without stepping on
> architected toes.
>
Re: Should Jackson-annotated POJOs stick around
Posted by Julian Hyde <jh...@apache.org>.
Here’s where I’m coming from. I’m not an expert on RPC, so maybe my enthusiasm for a clean, open, layered architecture is naivete as much as it is idealism. I probably mis-use terms “transport” and “wire protocol”.
I am very glad that genuine experts have joined this conversation (and since this is email, I should say that I mean that sincerely). Please take my words with a grain of salt but heed my overall message: that we should design this as a stack that can be re-used and evolve, not as a driver for a particular database and language binding.
It is also becoming clearer to me that at some point we should split Avatica into a project separate from Calcite. The thread about Calcite’s graduation is probably a better place for that discussion.
Julian
> On Aug 11, 2015, at 2:44 PM, Josh Elser <jo...@gmail.com> wrote:
>
> Ted Dunning wrote:
>> On Tue, Aug 11, 2015 at 1:14 PM, Josh Elser<jo...@gmail.com> wrote:
>>
>>> Andrew Purtell wrote:
>>>
>>>> That might be because protobuf documentation, and I'd assume accumulated
>>>> practice based upon it, warns against using generated pbuf objects
>>>> directly
>>>> as model classes. (See the "Protocol Buffers and O-O Design" callout on
>>>> https://developers.google.com/protocol-buffers/docs/javatutorial.)
>>>>
>>> Assuming that's the case, that makes sense. It was just not clear to me if
>>> Julian and I were just talking past each other or if there was some fallacy
>>> I was suggesting.
>>
>>
>> This differentiation between wire protocol and API is something that I have
>> seen repeatedly in ex-Googlers. I was a bit curious since it seemed nice to
>> have one definition for both levels.
>>
>> My opinion has verged to be 100% with the Google philosophy of separation
>> after watching how the MapR internals work. This kind of separation has
>> really paid off in many instances. Having too tight a lock between wire and
>> API would have been nearly disastrous for either comprehensibility of the
>> API or efficiency of the wire. I can't share specifics, but if second-hand
>> opinions are useful, you now have mine.
>>
>
> Absolutely, opinions are very useful here, Ted. Much appreciated. I'm trying to feel my way through the cleanest approach without stepping on architected toes.
Re: Should Jackson-annotated POJOs stick around
Posted by Josh Elser <jo...@gmail.com>.
Ted Dunning wrote:
> On Tue, Aug 11, 2015 at 1:14 PM, Josh Elser<jo...@gmail.com> wrote:
>
>> Andrew Purtell wrote:
>>
>>> That might be because protobuf documentation, and I'd assume accumulated
>>> practice based upon it, warns against using generated pbuf objects
>>> directly
>>> as model classes. (See the "Protocol Buffers and O-O Design" callout on
>>> https://developers.google.com/protocol-buffers/docs/javatutorial.)
>>>
>> Assuming that's the case, that makes sense. It was just not clear to me if
>> Julian and I were just talking past each other or if there was some fallacy
>> I was suggesting.
>
>
> This differentiation between wire protocol and API is something that I have
> seen repeatedly in ex-Googlers. I was a bit curious since it seemed nice to
> have one definition for both levels.
>
> My opinion has verged to be 100% with the Google philosophy of separation
> after watching how the MapR internals work. This kind of separation has
> really paid off in many instances. Having too tight a lock between wire and
> API would have been nearly disastrous for either comprehensibility of the
> API or efficiency of the wire. I can't share specifics, but if second-hand
> opinions are useful, you now have mine.
>
Absolutely, opinions are very useful here, Ted. Much appreciated. I'm
trying to feel my way through the cleanest approach without stepping on
architected toes.
Re: Should Jackson-annotated POJOs stick around
Posted by Ted Dunning <te...@gmail.com>.
On Tue, Aug 11, 2015 at 1:14 PM, Josh Elser <jo...@gmail.com> wrote:
> Andrew Purtell wrote:
>
>> That might be because protobuf documentation, and I'd assume accumulated
>> practice based upon it, warns against using generated pbuf objects
>> directly
>> as model classes. (See the "Protocol Buffers and O-O Design" callout on
>> https://developers.google.com/protocol-buffers/docs/javatutorial.)
>>
>
> Assuming that's the case, that makes sense. It was just not clear to me if
> Julian and I were just talking past each other or if there was some fallacy
> I was suggesting.
This differentiation between wire protocol and API is something that I have
seen repeatedly in ex-Googlers. I was a bit curious since it seemed nice to
have one definition for both levels.
My opinion has verged to be 100% with the Google philosophy of separation
after watching how the MapR internals work. This kind of separation has
really paid off in many instances. Having too tight a lock between wire and
API would have been nearly disastrous for either comprehensibility of the
API or efficiency of the wire. I can't share specifics, but if second-hand
opinions are useful, you now have mine.
Re: Should Jackson-annotated POJOs stick around
Posted by Andrew Purtell <ap...@apache.org>.
Watching CALCITE-840 with interest.
On Tue, Aug 11, 2015 at 1:14 PM, Josh Elser <jo...@gmail.com> wrote:
> Andrew Purtell wrote:
>
>> That might be because protobuf documentation, and I'd assume accumulated
>> practice based upon it, warns against using generated pbuf objects
>> directly
>> as model classes. (See the "Protocol Buffers and O-O Design" callout on
>> https://developers.google.com/protocol-buffers/docs/javatutorial.)
>>
>
> Assuming that's the case, that makes sense. It was just not clear to me if
> Julian and I were just talking past each other or if there was some fallacy
> I was suggesting.
>
> Maybe it's possible to wire up Jackson for emitting protobuf output as
>> another transport option?
>> https://github.com/FasterXML/jackson-dataformat-protobuf
>>
>
> Thanks for the link. After Paul's offer of help, I meant to look around to
> see what this interaction would actually look like. I have more reading to
> do.
>
>
> On Tue, Aug 11, 2015 at 11:47 AM, Josh Elser<jo...@gmail.com> wrote:
>>
>> > I'll admit that I don't understand why protocol buffers is being
>>> equated
>>> > with the transport only.
>>> >
>>> > The biggest reason I think Avatica can benefit from using software
>>> like
>>> > protobuf is that it makes handling a shift in the API substantially
>>> easier.
>>> > For example, what happens when a new field is added to a Request?
>>> What if
>>> > you receive a Request that doesn't have a field that you expected it
>>> to
>>> > (old client)? This is the problem that I'm trying to solve.
>>> Regardless of
>>> > whether this is coming in over the Java API or an HTTP connection, the
>>> > version of the Request or Response (and the actual attributes that it
>>> > contains) are near guaranteed to change.
>>> >
>>> > I don't really care what the bytes look like going over the wire.
>>> That's
>>> > just a side-effect to which my only concern is to meet any desires for
>>> > readability that may exist.
>>> >
>>> >
>>> > Does that make sense?
>>>
>>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Should Jackson-annotated POJOs stick around
Posted by Josh Elser <jo...@gmail.com>.
Andrew Purtell wrote:
> That might be because protobuf documentation, and I'd assume accumulated
> practice based upon it, warns against using generated pbuf objects directly
> as model classes. (See the "Protocol Buffers and O-O Design" callout on
> https://developers.google.com/protocol-buffers/docs/javatutorial.)
Assuming that's the case, that makes sense. It was just not clear to me
if Julian and I were just talking past each other or if there was some
fallacy I was suggesting.
> Maybe it's possible to wire up Jackson for emitting protobuf output as
> another transport option?
> https://github.com/FasterXML/jackson-dataformat-protobuf
Thanks for the link. After Paul's offer of help, I meant to look around
to see what this interaction would actually look like. I have more
reading to do.
> On Tue, Aug 11, 2015 at 11:47 AM, Josh Elser<jo...@gmail.com> wrote:
>
>> > I'll admit that I don't understand why protocol buffers is being equated
>> > with the transport only.
>> >
>> > The biggest reason I think Avatica can benefit from using software like
>> > protobuf is that it makes handling a shift in the API substantially easier.
>> > For example, what happens when a new field is added to a Request? What if
>> > you receive a Request that doesn't have a field that you expected it to
>> > (old client)? This is the problem that I'm trying to solve. Regardless of
>> > whether this is coming in over the Java API or an HTTP connection, the
>> > version of the Request or Response (and the actual attributes that it
>> > contains) are near guaranteed to change.
>> >
>> > I don't really care what the bytes look like going over the wire. That's
>> > just a side-effect to which my only concern is to meet any desires for
>> > readability that may exist.
>> >
>> >
>> > Does that make sense?
Re: Should Jackson-annotated POJOs stick around
Posted by Andrew Purtell <ap...@apache.org>.
That might be because protobuf documentation, and I'd assume accumulated
practice based upon it, warns against using generated pbuf objects directly
as model classes. (See the "Protocol Buffers and O-O Design" callout on
https://developers.google.com/protocol-buffers/docs/javatutorial.)
Maybe it's possible to wire up Jackson for emitting protobuf output as
another transport option?
https://github.com/FasterXML/jackson-dataformat-protobuf
On Tue, Aug 11, 2015 at 11:47 AM, Josh Elser <jo...@gmail.com> wrote:
> I'll admit that I don't understand why protocol buffers is being equated
> with the transport only.
>
> The biggest reason I think Avatica can benefit from using software like
> protobuf is that it makes handling a shift in the API substantially easier.
> For example, what happens when a new field is added to a Request? What if
> you receive a Request that doesn't have a field that you expected it to
> (old client)? This is the problem that I'm trying to solve. Regardless of
> whether this is coming in over the Java API or an HTTP connection, the
> version of the Request or Response (and the actual attributes that it
> contains) are near guaranteed to change.
>
> I don't really care what the bytes look like going over the wire. That's
> just a side-effect to which my only concern is to meet any desires for
> readability that may exist.
>
>
> Does that make sense?
>
> Julian Hyde wrote:
>
>> The conventional way of implementing a driver uses an end-to-end
>> design: "just use the MySQL driver and have a MySQL server on the
>> other end and it will just work".
>>
>> With Avatica I'm trying to introduce a layered approach (inspired by
>> the OSI model) where we specify what happens at several levels. We
>> specify a wire format, and we specify the Java API.
>>
>> An application doesn't have to to use all of the levels. Some
>> applications might want to use the Java API but plug in a different
>> transport. Other applications might want to build a C or Python API on
>> top of the wire format.
>>
>> My goal is to ensure that the driver stack can be re-used for
>> different database engines, and from different languages, and that a
>> driver in local mode is substantially the same as a driver in remote
>> mode.
>>
>> Notice I said "a wire format". I don't mind if there are other wire
>> formats. But I would be concerned if one transport muscled in and left
>> its mark up and down the whole stack.
>>
>>
>> On Tue, Aug 11, 2015 at 10:58 AM, Josh Elser<jo...@gmail.com>
>> wrote:
>>
>>> Ok, that's where the confusion is coming in :). I envision protocol
>>> buffer
>>> objects replacing the POJOs in both the wire api and the Meta API.
>>>
>>> Trying to do both seems without gain for me (again, aside from preserving
>>> backwards compatibility with existing releases). If that requires a
>>> different Meta interface+impl (or a translation layer), that's fine too
>>> --
>>> just extra work (and a slight performance hit on whichever is treated as
>>> the
>>> reference object).
>>>
>>> Does that make sense?
>>>
>>> Julian Hyde wrote:
>>>
>>>> Yes, the POJOs are needed. The service layer (which for a particular
>>>> client may or may not be backed by RPC) consists of methods that take
>>>> complex arguments and return complex results. Those arguments and
>>>> results are expressed as POJOs.
>>>>
>>>> One example:
>>>>
>>>> Frame fetch(StatementHandle h, List<TypedValue> parameterValues, long
>>>> offset,
>>>> int fetchMaxRowCount);
>>>>
>>>> class Frame {
>>>> public final long offset;
>>>> public final boolean done;
>>>> public final Iterable<Object> rows;
>>>> }
>>>>
>>>> public class TypedValue {
>>>> public final ColumnMetaData.Rep type;
>>>> public final Object value;
>>>> }
>>>>
>>>> The POJOs are definitely needed for the Java API Meta and appear in
>>>> some form by whatever the wire protocol is.
>>>>
>>>> Julian
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Aug 11, 2015 at 10:16 AM, Josh Elser<jo...@gmail.com>
>>>> wrote:
>>>>
>>>>> Pulling this out to email to avoid cluttering JIRA. I feel like I might
>>>>> not
>>>>> be on the same page.
>>>>>
>>>>> I see CALCITE-839 and CALCITE-840 being one in the same change, or at
>>>>> least
>>>>> the root cause being solved by it.
>>>>>
>>>>> Julian, I'm getting the impression that you envision protocol buffer
>>>>> encoding being just another option for encoding requests and responses.
>>>>> My
>>>>> opinion is that using protocol buffers to define these requests and
>>>>> responses completely invalidates the need to support these POJOs. These
>>>>> objects should be usable cross-language, so aside from support the
>>>>> releases
>>>>> of Calcite which shipped only these POJOs, I don't see a need to
>>>>> maintain
>>>>> them.
>>>>>
>>>>> I am admittedly hedging my bet that the PB devs will release a new
>>>>> version
>>>>> that has the advertised JSON-esque serialization format (instead of
>>>>> just
>>>>> a
>>>>> binary format). If this ultimately falls through, POJOs that just wrap
>>>>> the
>>>>> PB classes could also be done.
>>>>>
>>>>> I just wanted to make sure I'm not dancing by myself and that we're all
>>>>> in
>>>>> agreement on a general direction.
>>>>>
>>>>> -------- Original Message --------
>>>>> Subject: [jira] [Commented] (CALCITE-839) Remove Jackson annotations
>>>>> from
>>>>> POJO classes in Meta
>>>>> Date: Tue, 11 Aug 2015 05:41:45 +0000 (UTC)
>>>>> From: Julian Hyde (JIRA)<ji...@apache.org>
>>>>> Reply-To: dev@calcite.incubator.apache.org
>>>>> To: issues@calcite.incubator.apache.org
>>>>>
>>>>>
>>>>> [
>>>>>
>>>>>
>>>>> https://issues.apache.org/jira/browse/CALCITE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681279#comment-14681279
>>>>> ]
>>>>>
>>>>> Julian Hyde commented on CALCITE-839:
>>>>> -------------------------------------
>>>>>
>>>>> Well then, I've assigned CALCITE-840 to you.
>>>>>
>>>>> Remove Jackson annotations from POJO classes in Meta
>>>>>> ----------------------------------------------------
>>>>>>
>>>>>> Key: CALCITE-839
>>>>>> URL:
>>>>>> https://issues.apache.org/jira/browse/CALCITE-839
>>>>>> Project: Calcite
>>>>>> Issue Type: Bug
>>>>>> Components: avatica
>>>>>> Reporter: Julian Hyde
>>>>>> Assignee: Julian Hyde
>>>>>>
>>>>>> The Meta interface contains several POJO classes that represent RPC
>>>>>> requests or responses. Currently a few of those classes have Jackson
>>>>>> annotations such as @JsonCreator, @JsonProperty to help Jackson
>>>>>> serialize
>>>>>> the POJO to JSON and de-serialize from JSON to the object.
>>>>>> As [~ndimiduk] pointed out in
>>>>>>
>>>>>>
>>>>>> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201503.mbox/%3CCANZa=GvKGD+BKJ4+eJMuO6iVhS+OkGSkG1VwDAZCy-ZijyyyPw@mail.gmail.com%3E
>>>>>> these annotations are a "code smell" and should be removed. It makes
>>>>>> it
>>>>>> look
>>>>>> as if Jackson is the only possible transport, which is not the case.
>>>>>> We
>>>>>> can
>>>>>> continue to use Jackson as a transport, just specify the mappings
>>>>>> elsewhere,
>>>>>> not as annotations.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> This message was sent by Atlassian JIRA
>>>>> (v6.3.4#6332)
>>>>>
>>>>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Should Jackson-annotated POJOs stick around
Posted by Josh Elser <jo...@gmail.com>.
I'll admit that I don't understand why protocol buffers is being equated
with the transport only.
The biggest reason I think Avatica can benefit from using software like
protobuf is that it makes handling a shift in the API substantially
easier. For example, what happens when a new field is added to a
Request? What if you receive a Request that doesn't have a field that
you expected it to (old client)? This is the problem that I'm trying to
solve. Regardless of whether this is coming in over the Java API or an
HTTP connection, the version of the Request or Response (and the actual
attributes that it contains) are near guaranteed to change.
I don't really care what the bytes look like going over the wire. That's
just a side-effect to which my only concern is to meet any desires for
readability that may exist.
Does that make sense?
Julian Hyde wrote:
> The conventional way of implementing a driver uses an end-to-end
> design: "just use the MySQL driver and have a MySQL server on the
> other end and it will just work".
>
> With Avatica I'm trying to introduce a layered approach (inspired by
> the OSI model) where we specify what happens at several levels. We
> specify a wire format, and we specify the Java API.
>
> An application doesn't have to to use all of the levels. Some
> applications might want to use the Java API but plug in a different
> transport. Other applications might want to build a C or Python API on
> top of the wire format.
>
> My goal is to ensure that the driver stack can be re-used for
> different database engines, and from different languages, and that a
> driver in local mode is substantially the same as a driver in remote
> mode.
>
> Notice I said "a wire format". I don't mind if there are other wire
> formats. But I would be concerned if one transport muscled in and left
> its mark up and down the whole stack.
>
>
> On Tue, Aug 11, 2015 at 10:58 AM, Josh Elser<jo...@gmail.com> wrote:
>> Ok, that's where the confusion is coming in :). I envision protocol buffer
>> objects replacing the POJOs in both the wire api and the Meta API.
>>
>> Trying to do both seems without gain for me (again, aside from preserving
>> backwards compatibility with existing releases). If that requires a
>> different Meta interface+impl (or a translation layer), that's fine too --
>> just extra work (and a slight performance hit on whichever is treated as the
>> reference object).
>>
>> Does that make sense?
>>
>> Julian Hyde wrote:
>>> Yes, the POJOs are needed. The service layer (which for a particular
>>> client may or may not be backed by RPC) consists of methods that take
>>> complex arguments and return complex results. Those arguments and
>>> results are expressed as POJOs.
>>>
>>> One example:
>>>
>>> Frame fetch(StatementHandle h, List<TypedValue> parameterValues, long
>>> offset,
>>> int fetchMaxRowCount);
>>>
>>> class Frame {
>>> public final long offset;
>>> public final boolean done;
>>> public final Iterable<Object> rows;
>>> }
>>>
>>> public class TypedValue {
>>> public final ColumnMetaData.Rep type;
>>> public final Object value;
>>> }
>>>
>>> The POJOs are definitely needed for the Java API Meta and appear in
>>> some form by whatever the wire protocol is.
>>>
>>> Julian
>>>
>>>
>>>
>>>
>>> On Tue, Aug 11, 2015 at 10:16 AM, Josh Elser<jo...@gmail.com> wrote:
>>>> Pulling this out to email to avoid cluttering JIRA. I feel like I might
>>>> not
>>>> be on the same page.
>>>>
>>>> I see CALCITE-839 and CALCITE-840 being one in the same change, or at
>>>> least
>>>> the root cause being solved by it.
>>>>
>>>> Julian, I'm getting the impression that you envision protocol buffer
>>>> encoding being just another option for encoding requests and responses.
>>>> My
>>>> opinion is that using protocol buffers to define these requests and
>>>> responses completely invalidates the need to support these POJOs. These
>>>> objects should be usable cross-language, so aside from support the
>>>> releases
>>>> of Calcite which shipped only these POJOs, I don't see a need to maintain
>>>> them.
>>>>
>>>> I am admittedly hedging my bet that the PB devs will release a new
>>>> version
>>>> that has the advertised JSON-esque serialization format (instead of just
>>>> a
>>>> binary format). If this ultimately falls through, POJOs that just wrap
>>>> the
>>>> PB classes could also be done.
>>>>
>>>> I just wanted to make sure I'm not dancing by myself and that we're all
>>>> in
>>>> agreement on a general direction.
>>>>
>>>> -------- Original Message --------
>>>> Subject: [jira] [Commented] (CALCITE-839) Remove Jackson annotations from
>>>> POJO classes in Meta
>>>> Date: Tue, 11 Aug 2015 05:41:45 +0000 (UTC)
>>>> From: Julian Hyde (JIRA)<ji...@apache.org>
>>>> Reply-To: dev@calcite.incubator.apache.org
>>>> To: issues@calcite.incubator.apache.org
>>>>
>>>>
>>>> [
>>>>
>>>> https://issues.apache.org/jira/browse/CALCITE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681279#comment-14681279
>>>> ]
>>>>
>>>> Julian Hyde commented on CALCITE-839:
>>>> -------------------------------------
>>>>
>>>> Well then, I've assigned CALCITE-840 to you.
>>>>
>>>>> Remove Jackson annotations from POJO classes in Meta
>>>>> ----------------------------------------------------
>>>>>
>>>>> Key: CALCITE-839
>>>>> URL: https://issues.apache.org/jira/browse/CALCITE-839
>>>>> Project: Calcite
>>>>> Issue Type: Bug
>>>>> Components: avatica
>>>>> Reporter: Julian Hyde
>>>>> Assignee: Julian Hyde
>>>>>
>>>>> The Meta interface contains several POJO classes that represent RPC
>>>>> requests or responses. Currently a few of those classes have Jackson
>>>>> annotations such as @JsonCreator, @JsonProperty to help Jackson
>>>>> serialize
>>>>> the POJO to JSON and de-serialize from JSON to the object.
>>>>> As [~ndimiduk] pointed out in
>>>>>
>>>>> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201503.mbox/%3CCANZa=GvKGD+BKJ4+eJMuO6iVhS+OkGSkG1VwDAZCy-ZijyyyPw@mail.gmail.com%3E
>>>>> these annotations are a "code smell" and should be removed. It makes it
>>>>> look
>>>>> as if Jackson is the only possible transport, which is not the case. We
>>>>> can
>>>>> continue to use Jackson as a transport, just specify the mappings
>>>>> elsewhere,
>>>>> not as annotations.
>>>>
>>>>
>>>>
>>>> --
>>>> This message was sent by Atlassian JIRA
>>>> (v6.3.4#6332)
Re: Should Jackson-annotated POJOs stick around
Posted by Josh Elser <jo...@gmail.com>.
Looking at the link Andy sent out
(https://github.com/FasterXML/jackson-dataformat-protobuf), I have two
questions if you have the time:
1. "Experimental Jackson module" is that a concern for
stability/maintenance?
2. "Version 2 of protoc supported; parser supports v3 as well but
encoder/decoder does not." what does this mean WRT pbufv3? I couldn't
find details about what the lack of an encode/decoder really means if I
would try to do this.
Thanks!
Paul Brown wrote:
> Just to let you guys know that you have some Jackson folks lurking around
> on the list, so if there are things that we can do to help PB/JSON/other
> work smoothly, let us know, as we're actively thinking about the best ways
> to deal with multiple data protocols.
>
> —
> prb@mult.ifario.us | Multifarious, Inc. | http://mult.ifario.us/
>
Re: Should Jackson-annotated POJOs stick around
Posted by Paul Brown <pr...@mult.ifario.us>.
Just to let you guys know that you have some Jackson folks lurking around
on the list, so if there are things that we can do to help PB/JSON/other
work smoothly, let us know, as we're actively thinking about the best ways
to deal with multiple data protocols.
—
prb@mult.ifario.us | Multifarious, Inc. | http://mult.ifario.us/
On Tue, Aug 11, 2015 at 11:21 AM, Julian Hyde <jh...@apache.org> wrote:
> The conventional way of implementing a driver uses an end-to-end
> design: "just use the MySQL driver and have a MySQL server on the
> other end and it will just work".
>
> With Avatica I'm trying to introduce a layered approach (inspired by
> the OSI model) where we specify what happens at several levels. We
> specify a wire format, and we specify the Java API.
>
> An application doesn't have to to use all of the levels. Some
> applications might want to use the Java API but plug in a different
> transport. Other applications might want to build a C or Python API on
> top of the wire format.
>
> My goal is to ensure that the driver stack can be re-used for
> different database engines, and from different languages, and that a
> driver in local mode is substantially the same as a driver in remote
> mode.
>
> Notice I said "a wire format". I don't mind if there are other wire
> formats. But I would be concerned if one transport muscled in and left
> its mark up and down the whole stack.
>
>
> On Tue, Aug 11, 2015 at 10:58 AM, Josh Elser <jo...@gmail.com> wrote:
> > Ok, that's where the confusion is coming in :). I envision protocol
> buffer
> > objects replacing the POJOs in both the wire api and the Meta API.
> >
> > Trying to do both seems without gain for me (again, aside from preserving
> > backwards compatibility with existing releases). If that requires a
> > different Meta interface+impl (or a translation layer), that's fine too
> --
> > just extra work (and a slight performance hit on whichever is treated as
> the
> > reference object).
> >
> > Does that make sense?
> >
> > Julian Hyde wrote:
> >>
> >> Yes, the POJOs are needed. The service layer (which for a particular
> >> client may or may not be backed by RPC) consists of methods that take
> >> complex arguments and return complex results. Those arguments and
> >> results are expressed as POJOs.
> >>
> >> One example:
> >>
> >> Frame fetch(StatementHandle h, List<TypedValue> parameterValues, long
> >> offset,
> >> int fetchMaxRowCount);
> >>
> >> class Frame {
> >> public final long offset;
> >> public final boolean done;
> >> public final Iterable<Object> rows;
> >> }
> >>
> >> public class TypedValue {
> >> public final ColumnMetaData.Rep type;
> >> public final Object value;
> >> }
> >>
> >> The POJOs are definitely needed for the Java API Meta and appear in
> >> some form by whatever the wire protocol is.
> >>
> >> Julian
> >>
> >>
> >>
> >>
> >> On Tue, Aug 11, 2015 at 10:16 AM, Josh Elser<jo...@gmail.com>
> wrote:
> >>>
> >>> Pulling this out to email to avoid cluttering JIRA. I feel like I might
> >>> not
> >>> be on the same page.
> >>>
> >>> I see CALCITE-839 and CALCITE-840 being one in the same change, or at
> >>> least
> >>> the root cause being solved by it.
> >>>
> >>> Julian, I'm getting the impression that you envision protocol buffer
> >>> encoding being just another option for encoding requests and responses.
> >>> My
> >>> opinion is that using protocol buffers to define these requests and
> >>> responses completely invalidates the need to support these POJOs. These
> >>> objects should be usable cross-language, so aside from support the
> >>> releases
> >>> of Calcite which shipped only these POJOs, I don't see a need to
> maintain
> >>> them.
> >>>
> >>> I am admittedly hedging my bet that the PB devs will release a new
> >>> version
> >>> that has the advertised JSON-esque serialization format (instead of
> just
> >>> a
> >>> binary format). If this ultimately falls through, POJOs that just wrap
> >>> the
> >>> PB classes could also be done.
> >>>
> >>> I just wanted to make sure I'm not dancing by myself and that we're all
> >>> in
> >>> agreement on a general direction.
> >>>
> >>> -------- Original Message --------
> >>> Subject: [jira] [Commented] (CALCITE-839) Remove Jackson annotations
> from
> >>> POJO classes in Meta
> >>> Date: Tue, 11 Aug 2015 05:41:45 +0000 (UTC)
> >>> From: Julian Hyde (JIRA)<ji...@apache.org>
> >>> Reply-To: dev@calcite.incubator.apache.org
> >>> To: issues@calcite.incubator.apache.org
> >>>
> >>>
> >>> [
> >>>
> >>>
> https://issues.apache.org/jira/browse/CALCITE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681279#comment-14681279
> >>> ]
> >>>
> >>> Julian Hyde commented on CALCITE-839:
> >>> -------------------------------------
> >>>
> >>> Well then, I've assigned CALCITE-840 to you.
> >>>
> >>>> Remove Jackson annotations from POJO classes in Meta
> >>>> ----------------------------------------------------
> >>>>
> >>>> Key: CALCITE-839
> >>>> URL:
> https://issues.apache.org/jira/browse/CALCITE-839
> >>>> Project: Calcite
> >>>> Issue Type: Bug
> >>>> Components: avatica
> >>>> Reporter: Julian Hyde
> >>>> Assignee: Julian Hyde
> >>>>
> >>>> The Meta interface contains several POJO classes that represent RPC
> >>>> requests or responses. Currently a few of those classes have Jackson
> >>>> annotations such as @JsonCreator, @JsonProperty to help Jackson
> >>>> serialize
> >>>> the POJO to JSON and de-serialize from JSON to the object.
> >>>> As [~ndimiduk] pointed out in
> >>>>
> >>>>
> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201503.mbox/%3CCANZa=GvKGD+BKJ4+eJMuO6iVhS+OkGSkG1VwDAZCy-ZijyyyPw@mail.gmail.com%3E
> >>>> these annotations are a "code smell" and should be removed. It makes
> it
> >>>> look
> >>>> as if Jackson is the only possible transport, which is not the case.
> We
> >>>> can
> >>>> continue to use Jackson as a transport, just specify the mappings
> >>>> elsewhere,
> >>>> not as annotations.
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> This message was sent by Atlassian JIRA
> >>> (v6.3.4#6332)
>
Re: Should Jackson-annotated POJOs stick around
Posted by Julian Hyde <jh...@apache.org>.
The conventional way of implementing a driver uses an end-to-end
design: "just use the MySQL driver and have a MySQL server on the
other end and it will just work".
With Avatica I'm trying to introduce a layered approach (inspired by
the OSI model) where we specify what happens at several levels. We
specify a wire format, and we specify the Java API.
An application doesn't have to to use all of the levels. Some
applications might want to use the Java API but plug in a different
transport. Other applications might want to build a C or Python API on
top of the wire format.
My goal is to ensure that the driver stack can be re-used for
different database engines, and from different languages, and that a
driver in local mode is substantially the same as a driver in remote
mode.
Notice I said "a wire format". I don't mind if there are other wire
formats. But I would be concerned if one transport muscled in and left
its mark up and down the whole stack.
On Tue, Aug 11, 2015 at 10:58 AM, Josh Elser <jo...@gmail.com> wrote:
> Ok, that's where the confusion is coming in :). I envision protocol buffer
> objects replacing the POJOs in both the wire api and the Meta API.
>
> Trying to do both seems without gain for me (again, aside from preserving
> backwards compatibility with existing releases). If that requires a
> different Meta interface+impl (or a translation layer), that's fine too --
> just extra work (and a slight performance hit on whichever is treated as the
> reference object).
>
> Does that make sense?
>
> Julian Hyde wrote:
>>
>> Yes, the POJOs are needed. The service layer (which for a particular
>> client may or may not be backed by RPC) consists of methods that take
>> complex arguments and return complex results. Those arguments and
>> results are expressed as POJOs.
>>
>> One example:
>>
>> Frame fetch(StatementHandle h, List<TypedValue> parameterValues, long
>> offset,
>> int fetchMaxRowCount);
>>
>> class Frame {
>> public final long offset;
>> public final boolean done;
>> public final Iterable<Object> rows;
>> }
>>
>> public class TypedValue {
>> public final ColumnMetaData.Rep type;
>> public final Object value;
>> }
>>
>> The POJOs are definitely needed for the Java API Meta and appear in
>> some form by whatever the wire protocol is.
>>
>> Julian
>>
>>
>>
>>
>> On Tue, Aug 11, 2015 at 10:16 AM, Josh Elser<jo...@gmail.com> wrote:
>>>
>>> Pulling this out to email to avoid cluttering JIRA. I feel like I might
>>> not
>>> be on the same page.
>>>
>>> I see CALCITE-839 and CALCITE-840 being one in the same change, or at
>>> least
>>> the root cause being solved by it.
>>>
>>> Julian, I'm getting the impression that you envision protocol buffer
>>> encoding being just another option for encoding requests and responses.
>>> My
>>> opinion is that using protocol buffers to define these requests and
>>> responses completely invalidates the need to support these POJOs. These
>>> objects should be usable cross-language, so aside from support the
>>> releases
>>> of Calcite which shipped only these POJOs, I don't see a need to maintain
>>> them.
>>>
>>> I am admittedly hedging my bet that the PB devs will release a new
>>> version
>>> that has the advertised JSON-esque serialization format (instead of just
>>> a
>>> binary format). If this ultimately falls through, POJOs that just wrap
>>> the
>>> PB classes could also be done.
>>>
>>> I just wanted to make sure I'm not dancing by myself and that we're all
>>> in
>>> agreement on a general direction.
>>>
>>> -------- Original Message --------
>>> Subject: [jira] [Commented] (CALCITE-839) Remove Jackson annotations from
>>> POJO classes in Meta
>>> Date: Tue, 11 Aug 2015 05:41:45 +0000 (UTC)
>>> From: Julian Hyde (JIRA)<ji...@apache.org>
>>> Reply-To: dev@calcite.incubator.apache.org
>>> To: issues@calcite.incubator.apache.org
>>>
>>>
>>> [
>>>
>>> https://issues.apache.org/jira/browse/CALCITE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681279#comment-14681279
>>> ]
>>>
>>> Julian Hyde commented on CALCITE-839:
>>> -------------------------------------
>>>
>>> Well then, I've assigned CALCITE-840 to you.
>>>
>>>> Remove Jackson annotations from POJO classes in Meta
>>>> ----------------------------------------------------
>>>>
>>>> Key: CALCITE-839
>>>> URL: https://issues.apache.org/jira/browse/CALCITE-839
>>>> Project: Calcite
>>>> Issue Type: Bug
>>>> Components: avatica
>>>> Reporter: Julian Hyde
>>>> Assignee: Julian Hyde
>>>>
>>>> The Meta interface contains several POJO classes that represent RPC
>>>> requests or responses. Currently a few of those classes have Jackson
>>>> annotations such as @JsonCreator, @JsonProperty to help Jackson
>>>> serialize
>>>> the POJO to JSON and de-serialize from JSON to the object.
>>>> As [~ndimiduk] pointed out in
>>>>
>>>> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201503.mbox/%3CCANZa=GvKGD+BKJ4+eJMuO6iVhS+OkGSkG1VwDAZCy-ZijyyyPw@mail.gmail.com%3E
>>>> these annotations are a "code smell" and should be removed. It makes it
>>>> look
>>>> as if Jackson is the only possible transport, which is not the case. We
>>>> can
>>>> continue to use Jackson as a transport, just specify the mappings
>>>> elsewhere,
>>>> not as annotations.
>>>
>>>
>>>
>>>
>>> --
>>> This message was sent by Atlassian JIRA
>>> (v6.3.4#6332)
Re: Should Jackson-annotated POJOs stick around
Posted by Josh Elser <jo...@gmail.com>.
Ok, that's where the confusion is coming in :). I envision protocol
buffer objects replacing the POJOs in both the wire api and the Meta API.
Trying to do both seems without gain for me (again, aside from
preserving backwards compatibility with existing releases). If that
requires a different Meta interface+impl (or a translation layer),
that's fine too -- just extra work (and a slight performance hit on
whichever is treated as the reference object).
Does that make sense?
Julian Hyde wrote:
> Yes, the POJOs are needed. The service layer (which for a particular
> client may or may not be backed by RPC) consists of methods that take
> complex arguments and return complex results. Those arguments and
> results are expressed as POJOs.
>
> One example:
>
> Frame fetch(StatementHandle h, List<TypedValue> parameterValues, long offset,
> int fetchMaxRowCount);
>
> class Frame {
> public final long offset;
> public final boolean done;
> public final Iterable<Object> rows;
> }
>
> public class TypedValue {
> public final ColumnMetaData.Rep type;
> public final Object value;
> }
>
> The POJOs are definitely needed for the Java API Meta and appear in
> some form by whatever the wire protocol is.
>
> Julian
>
>
>
>
> On Tue, Aug 11, 2015 at 10:16 AM, Josh Elser<jo...@gmail.com> wrote:
>> Pulling this out to email to avoid cluttering JIRA. I feel like I might not
>> be on the same page.
>>
>> I see CALCITE-839 and CALCITE-840 being one in the same change, or at least
>> the root cause being solved by it.
>>
>> Julian, I'm getting the impression that you envision protocol buffer
>> encoding being just another option for encoding requests and responses. My
>> opinion is that using protocol buffers to define these requests and
>> responses completely invalidates the need to support these POJOs. These
>> objects should be usable cross-language, so aside from support the releases
>> of Calcite which shipped only these POJOs, I don't see a need to maintain
>> them.
>>
>> I am admittedly hedging my bet that the PB devs will release a new version
>> that has the advertised JSON-esque serialization format (instead of just a
>> binary format). If this ultimately falls through, POJOs that just wrap the
>> PB classes could also be done.
>>
>> I just wanted to make sure I'm not dancing by myself and that we're all in
>> agreement on a general direction.
>>
>> -------- Original Message --------
>> Subject: [jira] [Commented] (CALCITE-839) Remove Jackson annotations from
>> POJO classes in Meta
>> Date: Tue, 11 Aug 2015 05:41:45 +0000 (UTC)
>> From: Julian Hyde (JIRA)<ji...@apache.org>
>> Reply-To: dev@calcite.incubator.apache.org
>> To: issues@calcite.incubator.apache.org
>>
>>
>> [
>> https://issues.apache.org/jira/browse/CALCITE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681279#comment-14681279
>> ]
>>
>> Julian Hyde commented on CALCITE-839:
>> -------------------------------------
>>
>> Well then, I've assigned CALCITE-840 to you.
>>
>>> Remove Jackson annotations from POJO classes in Meta
>>> ----------------------------------------------------
>>>
>>> Key: CALCITE-839
>>> URL: https://issues.apache.org/jira/browse/CALCITE-839
>>> Project: Calcite
>>> Issue Type: Bug
>>> Components: avatica
>>> Reporter: Julian Hyde
>>> Assignee: Julian Hyde
>>>
>>> The Meta interface contains several POJO classes that represent RPC
>>> requests or responses. Currently a few of those classes have Jackson
>>> annotations such as @JsonCreator, @JsonProperty to help Jackson serialize
>>> the POJO to JSON and de-serialize from JSON to the object.
>>> As [~ndimiduk] pointed out in
>>> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201503.mbox/%3CCANZa=GvKGD+BKJ4+eJMuO6iVhS+OkGSkG1VwDAZCy-ZijyyyPw@mail.gmail.com%3E
>>> these annotations are a "code smell" and should be removed. It makes it look
>>> as if Jackson is the only possible transport, which is not the case. We can
>>> continue to use Jackson as a transport, just specify the mappings elsewhere,
>>> not as annotations.
>>
>>
>>
>> --
>> This message was sent by Atlassian JIRA
>> (v6.3.4#6332)
Re: Should Jackson-annotated POJOs stick around
Posted by Julian Hyde <jh...@apache.org>.
Yes, the POJOs are needed. The service layer (which for a particular
client may or may not be backed by RPC) consists of methods that take
complex arguments and return complex results. Those arguments and
results are expressed as POJOs.
One example:
Frame fetch(StatementHandle h, List<TypedValue> parameterValues, long offset,
int fetchMaxRowCount);
class Frame {
public final long offset;
public final boolean done;
public final Iterable<Object> rows;
}
public class TypedValue {
public final ColumnMetaData.Rep type;
public final Object value;
}
The POJOs are definitely needed for the Java API Meta and appear in
some form by whatever the wire protocol is.
Julian
On Tue, Aug 11, 2015 at 10:16 AM, Josh Elser <jo...@gmail.com> wrote:
> Pulling this out to email to avoid cluttering JIRA. I feel like I might not
> be on the same page.
>
> I see CALCITE-839 and CALCITE-840 being one in the same change, or at least
> the root cause being solved by it.
>
> Julian, I'm getting the impression that you envision protocol buffer
> encoding being just another option for encoding requests and responses. My
> opinion is that using protocol buffers to define these requests and
> responses completely invalidates the need to support these POJOs. These
> objects should be usable cross-language, so aside from support the releases
> of Calcite which shipped only these POJOs, I don't see a need to maintain
> them.
>
> I am admittedly hedging my bet that the PB devs will release a new version
> that has the advertised JSON-esque serialization format (instead of just a
> binary format). If this ultimately falls through, POJOs that just wrap the
> PB classes could also be done.
>
> I just wanted to make sure I'm not dancing by myself and that we're all in
> agreement on a general direction.
>
> -------- Original Message --------
> Subject: [jira] [Commented] (CALCITE-839) Remove Jackson annotations from
> POJO classes in Meta
> Date: Tue, 11 Aug 2015 05:41:45 +0000 (UTC)
> From: Julian Hyde (JIRA) <ji...@apache.org>
> Reply-To: dev@calcite.incubator.apache.org
> To: issues@calcite.incubator.apache.org
>
>
> [
> https://issues.apache.org/jira/browse/CALCITE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681279#comment-14681279
> ]
>
> Julian Hyde commented on CALCITE-839:
> -------------------------------------
>
> Well then, I've assigned CALCITE-840 to you.
>
>> Remove Jackson annotations from POJO classes in Meta
>> ----------------------------------------------------
>>
>> Key: CALCITE-839
>> URL: https://issues.apache.org/jira/browse/CALCITE-839
>> Project: Calcite
>> Issue Type: Bug
>> Components: avatica
>> Reporter: Julian Hyde
>> Assignee: Julian Hyde
>>
>> The Meta interface contains several POJO classes that represent RPC
>> requests or responses. Currently a few of those classes have Jackson
>> annotations such as @JsonCreator, @JsonProperty to help Jackson serialize
>> the POJO to JSON and de-serialize from JSON to the object.
>> As [~ndimiduk] pointed out in
>> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201503.mbox/%3CCANZa=GvKGD+BKJ4+eJMuO6iVhS+OkGSkG1VwDAZCy-ZijyyyPw@mail.gmail.com%3E
>> these annotations are a "code smell" and should be removed. It makes it look
>> as if Jackson is the only possible transport, which is not the case. We can
>> continue to use Jackson as a transport, just specify the mappings elsewhere,
>> not as annotations.
>
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)