You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Justin Ross <ju...@gmail.com> on 2015/01/28 21:36:08 UTC
Proton API layout proposal
https://cwiki.apache.org/confluence/display/qpid/Proton+API+layout+proposal
Please take a look. The intention:
- Organize the API consistently across languages, allowing for deliberate
exceptions
- Put more frequently used items closer to hand, and less frequently used
ones out of the way
- Put like things together in a named group
This scheme is now in use to some extent by the C and Python variants, but
there are some possibly non-semantic variations that I think are worth
ironing out:
- Python is using utils, not util. I don't care whether we go util or
utils, but I'd very much like to avoid it being half one way and half the
other. If we decide utils, I'll update the wiki page.
- C is using reactor, not reactors. Is the idea that it will have only
one reactor implementation?
Justin
Re: Proton API layout proposal
Posted by Gordon Sim <gs...@redhat.com>.
On 02/18/2015 04:04 PM, Justin Ross wrote:
> Right now Container is inside proton.reactor. Should it go into the core
> classes (proton) instead?
Not unless Reactor also went into core. I don't think you want anything
in core that depends on something outside. Container is simply an
extension of Reactor.
> It's notionally a peer to the endpoint types,
> and going by the examples, it has primary significance for users.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton API layout proposal
Posted by Justin Ross <ju...@gmail.com>.
Right now Container is inside proton.reactor. Should it go into the core
classes (proton) instead? It's notionally a peer to the endpoint types,
and going by the examples, it has primary significance for users.
Re: Proton API layout proposal
Posted by Gordon Sim <gs...@redhat.com>.
On 02/18/2015 04:18 PM, Gordon Sim wrote:
> On 02/18/2015 03:44 PM, Rafael Schloming wrote:
>> If we didn't already use the contrib convention I wouldn't actually care
>> all that much, but I feel like we should at least be consistent within
>> our
>> own codebase, so if we introduce python.extras, that's pretty much
>> pushing
>> for a rename of the java contrib stuff which seems like unnecessary churn
>> at this point.
>
> I don't think that is the same at all. The current contrib structure is
> a top level directory under which you have proton-jms and
> proton-hawtdispatch.
Putting the point more clearly, the code in these two components doesn't
include 'contrib' in anyway in their namespace. That is simply the name
of the top level directory in which they are currently housed.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton API layout proposal
Posted by Gordon Sim <gs...@redhat.com>.
On 02/18/2015 05:56 PM, Rafael Schloming wrote:
> On Wed, Feb 18, 2015 at 11:18 AM, Gordon Sim <gs...@redhat.com> wrote:
>
>> On 02/18/2015 03:44 PM, Rafael Schloming wrote:
>>
>>> I can't speak to the logic, but I believe the use of contrib to signal
>>> pretty much *exactly* what we are talking about here is a pretty common
>>> convention. There's a post here that describes django.contrib in much the
>>> same terms:
>>>
>>> http://jacobian.org/writing/what-is-django-contrib/
>>>
>>
>> I think that post describes the criteria for accepting something into the
>> 'blessed' contrib namespace. They don't describe why they called it
>> contrib. They describe it as 'extra, optional, tools' or in the link above
>> 'optional, de facto standard implementations of common patterns'.
>>
>
> No, and that's not why I referred to it. I assume the origin of calling it
> contrib started back somewhere in the annals of unix history, and is likely
> not super relevant to how it is actually understood today (like /etc, /opt,
> and so forth). That doesn't change the fact that at least among some people
> it has a pretty commonly understood meaning that is a good match for our
> purpose.
I am not one of those people, but perhaps I just lack some cultural
context or other. I don't think its a unix convention though.
[...]
>> I very much dislike it. However if you are adamant on that name, would
>> messenger be relocated there also?
>
>
> I'm actually not at all adamant on the name, however I am pretty adamant
> about consistency and avoiding churn. Regardless of what terminology we
> choose, I think the important thing here is to clearly communicate what the
> term implies. The post I pointed to actually does a pretty good job of
> this, and I think having two separate terms to mean the same thing is
> strictly worse for users than having one term. If you feel that using the
> term "extras" instead of "contrib" is going to improve our user experience,
> and you want to put the time in to change our existing usage of contrib
> over to extras, then I have no objection. To be perfectly clear though, I
> do object to introducing the term extras, defining it to mean almost the
> same thing as contrib, and ignoring the existing usage of contrib. I think
> this just creates more confusion.
The existing 'usage of contrib' appears to simply be the directory name,
and two references to that from the top level pom.xml, its not sued in
anything most users will even see, so the 'churn' thing seems somewhat
overstated.
I don't find the consistency argument terribly convincing either, since
what is being proposed as I understand it - a contrib module for
undefined 'stuff' within the proton package - seems to me to be
inconsistent with existing usage anyway.
As for communication as to what the term implies, the contrib directory
was requested in https://issues.apache.org/jira/browse/PROTON-140.
Unfortunately the irc snippet included doesn't provide much insight into
the motivation and I haven't been able to find anything about the
described F2F conversation either.
In the one thread I was able to find on the subject you state[1]:
The proton-j/contrib directory was created to hold
glue/integration with established third-party libraries
/apis (specifically jms and hawtdispatch). The fact that
hawtdispatch and jms don't want a proton dependency, and
proton doesn't want a jms or hawtdispatch dependency kind
of forces the glue/integration to sit somewhere separate
from each, and since it is kind of silly to have an
entirely separate project just for that glue/integration
code, creating a proton-j/contrib made sense.
The things we are talking about are not 'glue/integration with
established third-party libraries/apis' nor do they have extra
dependencies.
However this is really just saying that I find the arguments for contrib
unconvincing. Since you clearly don't like extras - and for that matter,
neither do I - let's come up with a name that is a more functional
description of the classes in question.
What about 'sync' or 'synchronous'?
[1]
http://mail-archives.apache.org/mod_mbox/qpid-proton/201212.mbox/%3CCACL1BnDrkdV07RnBzv7EWgMSuE4FtFr6SFNFxz52y8wLrVPxiw@mail.gmail.com%3E
> Regarding where messenger should be located, it depends on whether you
> consider it "optional".
It clearly *is* optional (under the usual definition of that term).
Would you consider it non-optional, i.e. required?
> If so, then I would actually say it's analogous to
> the hawtdispatch and jms examples, and build-wise it would be completely
> consistent to put it alongside them, particularly if that would enforce the
> layering constraint. I would leave the actual package name though since
> just like the hawtdispatch/jms modules it has enough of a well defined name
> and purpose that it doesn't really need to be bucketed from an API
> perspective.
It is currently in the proton package, so I think you would have to
change that if you moved it into a different part of the tree.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton API layout proposal
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Wed, Feb 18, 2015 at 11:18 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 02/18/2015 03:44 PM, Rafael Schloming wrote:
>
>> I can't speak to the logic, but I believe the use of contrib to signal
>> pretty much *exactly* what we are talking about here is a pretty common
>> convention. There's a post here that describes django.contrib in much the
>> same terms:
>>
>> http://jacobian.org/writing/what-is-django-contrib/
>>
>
> I think that post describes the criteria for accepting something into the
> 'blessed' contrib namespace. They don't describe why they called it
> contrib. They describe it as 'extra, optional, tools' or in the link above
> 'optional, de facto standard implementations of common patterns'.
>
No, and that's not why I referred to it. I assume the origin of calling it
contrib started back somewhere in the annals of unix history, and is likely
not super relevant to how it is actually understood today (like /etc, /opt,
and so forth). That doesn't change the fact that at least among some people
it has a pretty commonly understood meaning that is a good match for our
purpose.
> So extras or indeed optional would seem equally appropriate and to me,
> less confusing.
>
> If we didn't already use the contrib convention I wouldn't actually care
>> all that much, but I feel like we should at least be consistent within our
>> own codebase, so if we introduce python.extras, that's pretty much pushing
>> for a rename of the java contrib stuff which seems like unnecessary churn
>> at this point.
>>
>
> I don't think that is the same at all. The current contrib structure is a
> top level directory under which you have proton-jms and proton-hawtdispatch.
>
I think the use of the term contrib as a bucket and the criteria that is
applied to what goes into that bucket is (or should be) the same. The fact
that each of the units currently in the bucket have something more specific
that makes sense as a top level package name doesn't really change that,
the term is being used in the same way, just for adding structure in the
source tree rather than for adding structure in an API package.
>
> I do get the impression that there is possibly an unspoken bias against
>> contrib,
>>
>
> Unspoken? I said quite clearly I don't like it. To me it has no more
> meaning than say extras or optional, and the actual meaning of the word
> doesn't in my view match what we are talking about.
>
Sorry, my comment was unclear. I was actually referring to Justin's
implication that putting something in contrib is somehow disowning it, not
commenting on your dislike of the term.
>
> i.e. that's just a bucket to stick unwanted contributions, but I
>> really don't think we should view it that way. I think the comments in the
>> post above about it only containing best practices and what not are
>> something we should take to heart.
>>
>
> I very much dislike it. However if you are adamant on that name, would
> messenger be relocated there also?
I'm actually not at all adamant on the name, however I am pretty adamant
about consistency and avoiding churn. Regardless of what terminology we
choose, I think the important thing here is to clearly communicate what the
term implies. The post I pointed to actually does a pretty good job of
this, and I think having two separate terms to mean the same thing is
strictly worse for users than having one term. If you feel that using the
term "extras" instead of "contrib" is going to improve our user experience,
and you want to put the time in to change our existing usage of contrib
over to extras, then I have no objection. To be perfectly clear though, I
do object to introducing the term extras, defining it to mean almost the
same thing as contrib, and ignoring the existing usage of contrib. I think
this just creates more confusion.
Regarding where messenger should be located, it depends on whether you
consider it "optional". If so, then I would actually say it's analogous to
the hawtdispatch and jms examples, and build-wise it would be completely
consistent to put it alongside them, particularly if that would enforce the
layering constraint. I would leave the actual package name though since
just like the hawtdispatch/jms modules it has enough of a well defined name
and purpose that it doesn't really need to be bucketed from an API
perspective.
--Rafael
Re: Proton API layout proposal
Posted by Gordon Sim <gs...@redhat.com>.
On 02/18/2015 03:44 PM, Rafael Schloming wrote:
> I can't speak to the logic, but I believe the use of contrib to signal
> pretty much *exactly* what we are talking about here is a pretty common
> convention. There's a post here that describes django.contrib in much the
> same terms:
>
> http://jacobian.org/writing/what-is-django-contrib/
I think that post describes the criteria for accepting something into
the 'blessed' contrib namespace. They don't describe why they called it
contrib. They describe it as 'extra, optional, tools' or in the link
above 'optional, de facto standard implementations of common patterns'.
So extras or indeed optional would seem equally appropriate and to me,
less confusing.
> If we didn't already use the contrib convention I wouldn't actually care
> all that much, but I feel like we should at least be consistent within our
> own codebase, so if we introduce python.extras, that's pretty much pushing
> for a rename of the java contrib stuff which seems like unnecessary churn
> at this point.
I don't think that is the same at all. The current contrib structure is
a top level directory under which you have proton-jms and
proton-hawtdispatch.
> I do get the impression that there is possibly an unspoken bias against
> contrib,
Unspoken? I said quite clearly I don't like it. To me it has no more
meaning than say extras or optional, and the actual meaning of the word
doesn't in my view match what we are talking about.
> i.e. that's just a bucket to stick unwanted contributions, but I
> really don't think we should view it that way. I think the comments in the
> post above about it only containing best practices and what not are
> something we should take to heart.
I very much dislike it. However if you are adamant on that name, would
messenger be relocated there also?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton API layout proposal
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Wed, Feb 18, 2015 at 9:52 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 02/18/2015 02:28 PM, Rafael Schloming wrote:
>
>> On Wed, Feb 18, 2015 at 4:18 AM, Gordon Sim <gs...@redhat.com> wrote:
>>
>>> To amplify a little, the point was that the two things currently in the
>>> utils module are ways of adapting the reactive, non-blocking,
>>> event-driven
>>> style to some other style (messenger is in my view a similar sort of
>>> thing). Though it is certainly more narrow, I think its also more
>>> helpful.
>>>
>>> The other aspect to these is that they aren't yet considered as
>>> fully-baked as the reactive core. I certainly don't object to them being
>>> in
>>> an extras namespace to denote that, until we are more comfortable we have
>>> the interfaces right. You could also however indicate that through
>>> documentation or some 'experimental' annotation.
>>>
>>
>>
>> I agree we don't necessarily need to signal bakedness in the package name,
>> I think it's good to think of it as an orthogonal dimension. I'm not quite
>> sure I follow the reasoning for proton.adapters though. That sounds to me
>> like somewhere we'd put stuff for integrations, e.g. maybe the tornado
>> stuff.
>>
>
> I see these things as providing an alternative API on top of the reactive
> style API. Which to me is what an adapter does.
>
> To me the tornado integration is more of a plugin.
>
> I'm not adamant about the 'adapters' name though, just saying its clear to
> me(!). I really don't like 'contrib' though. For one thing it makes no
> logical sense to me. How are things more 'contributed' than anything else?
> I think 'extras' is actually a clearer way to describe the layering you
> mention. I don't mind that, though it is a little vague. (I think an
> adapter is also clearly layered).
I can't speak to the logic, but I believe the use of contrib to signal
pretty much *exactly* what we are talking about here is a pretty common
convention. There's a post here that describes django.contrib in much the
same terms:
http://jacobian.org/writing/what-is-django-contrib/
If we didn't already use the contrib convention I wouldn't actually care
all that much, but I feel like we should at least be consistent within our
own codebase, so if we introduce python.extras, that's pretty much pushing
for a rename of the java contrib stuff which seems like unnecessary churn
at this point.
I do get the impression that there is possibly an unspoken bias against
contrib, i.e. that's just a bucket to stick unwanted contributions, but I
really don't think we should view it that way. I think the comments in the
post above about it only containing best practices and what not are
something we should take to heart.
--Rafael
Re: Proton API layout proposal
Posted by Gordon Sim <gs...@redhat.com>.
On 02/18/2015 02:28 PM, Rafael Schloming wrote:
> On Wed, Feb 18, 2015 at 4:18 AM, Gordon Sim <gs...@redhat.com> wrote:
>> To amplify a little, the point was that the two things currently in the
>> utils module are ways of adapting the reactive, non-blocking, event-driven
>> style to some other style (messenger is in my view a similar sort of
>> thing). Though it is certainly more narrow, I think its also more helpful.
>>
>> The other aspect to these is that they aren't yet considered as
>> fully-baked as the reactive core. I certainly don't object to them being in
>> an extras namespace to denote that, until we are more comfortable we have
>> the interfaces right. You could also however indicate that through
>> documentation or some 'experimental' annotation.
>
>
> I agree we don't necessarily need to signal bakedness in the package name,
> I think it's good to think of it as an orthogonal dimension. I'm not quite
> sure I follow the reasoning for proton.adapters though. That sounds to me
> like somewhere we'd put stuff for integrations, e.g. maybe the tornado
> stuff.
I see these things as providing an alternative API on top of the
reactive style API. Which to me is what an adapter does.
To me the tornado integration is more of a plugin.
I'm not adamant about the 'adapters' name though, just saying its clear
to me(!). I really don't like 'contrib' though. For one thing it makes
no logical sense to me. How are things more 'contributed' than anything
else? I think 'extras' is actually a clearer way to describe the
layering you mention. I don't mind that, though it is a little vague. (I
think an adapter is also clearly layered).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton API layout proposal
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Wed, Feb 18, 2015 at 10:53 AM, Justin Ross <ju...@gmail.com> wrote:
> On Wed, Feb 18, 2015 at 9:28 AM, Rafael Schloming <rh...@alum.mit.edu>
> wrote:
>
> > A couple of comments in no particular order...
> >
> > The "Core user model and uitility classes" description could be
> > misinterpreted on a quick read. Maybe go with "Core protocol model..." or
> > something like that? (Really all these APIs are for users, so it seems
> > weird to mention it.) Also the utility classes mentioned here sound like
> > they overlap with the util package. What is that actually referring to?
> >
>
> Yes, good point. I'll go with protocol instead.
>
> If I could, I'd make util go away. It's for low-level things (not Proton
> extensions, but language library extensions) that we wish to make public.
> The best example we currently have is URL.
>
I actually was asking about the utility classes refered to in your "Core
user model and utility classes". It sounds like there are utility classes
directly in the proton package and utility classes in proton.util, which I
don't think is actually the case.
> > I like the proton.security addition, I think that makes sense as security
> > is generally handled in one contained place in any application.
> >
> > I'm not so hot on the types subpackage. I don't think it actually buys
> us a
> > lot in terms of uncluttering core since there are only a few very simple
> > things in that category. It basically consists of about 5 or so pure
> > constructors that extend builtin python types with extra information,
> e.g.
> > timestamp which is just a subclass of long that gets encoded on the wire
> as
> > a timestamp. It's implementation is literally just this[1], as you can
> see
> > it adds very little surface area to the API. The other items in this
> > category are similarly simple. Conceptually these types really are part
> of
> > the core protocol model, and unlike in the case of security, if your app
> > does need to deal with timestamps or unsigned numbers or whatever, it is
> > pretty likely that it will need to do it all over the place which means
> > that you basically always need to do two imports in order to get the full
> > protocol model.
> >
>
>
> http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-types-v1.0-os.html#section-primitive-type-definitions
>
> The number of AMQP primitive types is comparable to the number of endpoint
> types. A given language may not need all the types, but it will likely
> need more than half of them. Some of these will not find frequent use,
> such as the decimal types.
>
> I think it is a source of clutter (the implementation size is not a factor
> here; instead it's the number of things in a namespace). The grouping has
> the additional benefit of clarifying that these are all things of one sort.
>
> Are there particular types that you think will find frequent use and should
> be promoted to the "core" level? I too would like to consolidate the
> imports, but I disagree about the scale of the clutter problem.
>
If you're talking about the java link you reference below, that's not a
good way to measure the clutter. It includes all classes in all the sub
packages, so all the types would be there regardless of whether they are in
a sub package or not. I think the situation with clutter in the proton
package in python is probably significantly improved now since we got rid
of Driver and some other deprecated stuff.
>
> Gordon's python examples don't seem to import any types. For most higher
> level languages, I would hope this would be the case.
>
>From a quick scan it looks like all the examples are using u"Hello World"
as the message content. If we were to add in examples of sending more
structured data, then your impression of common usage might change.
Based on some use I've made of the API for implementing a message based
protocol on top of AMQP, timestamp is pretty common. I suspect there are
lots of scenarios where symbol and array would also be quite important. I
know it's tempting to ignore arrays since they don't map well to a builtin
python type, but arrays in particular can save you *significant* encoding
overhead, e.g. sending an array of integers will save you 25% encoding
overhead relative to sending a list of integers. This would be critical in
a bunch of different scenarios, e.g. any sort of numeric computing with
vectors and matrices, any sort of monitoring involving time series data. A
lot those scenarios might also care about signed vs unsigned numbers as
well as the actual precision of the value. Another quite general case is
anything wanting to interop with endpoints in other languages. In those
cases it may be necessary to be more explicit about the types you put into
messages.
I really do think some "real world usage" will point to all those types
living together. In my experience using the API so far, you basically
interact with Connection, Session, Link, Delivery, Message, and the
specific set of types you need to represent the content in your message,
and actually with some of the nice defaults in place, you don't even need
to think about Connection, Session, Link, or Delivery if you don't want to.
That means for a lot of cases Message pus the types is pretty much all you
care about, so having them in separate packages is kind of unfortunate.
Even including all the endpoint types, I don't think the list above should
be too long and cluttered for a single package. If there are other things
cluttering up the package I would look at what else we could clean up
before pushing types into a sub package. This is why I like your security
proposal as it gets the core package closer to just the really core stuff.
--Rafael
Re: Proton API layout proposal
Posted by Justin Ross <ju...@gmail.com>.
On Wed, Feb 18, 2015 at 10:53 AM, Justin Ross <ju...@gmail.com> wrote:
>
>
> On Wed, Feb 18, 2015 at 9:28 AM, Rafael Schloming <rh...@alum.mit.edu>
> wrote:
>
>> A couple of comments in no particular order...
>>
>> The "Core user model and uitility classes" description could be
>> misinterpreted on a quick read. Maybe go with "Core protocol model..." or
>> something like that? (Really all these APIs are for users, so it seems
>> weird to mention it.) Also the utility classes mentioned here sound like
>> they overlap with the util package. What is that actually referring to?
>>
>
> Yes, good point. I'll go with protocol instead.
>
> If I could, I'd make util go away. It's for low-level things (not Proton
> extensions, but language library extensions) that we wish to make public.
> The best example we currently have is URL.
>
I should say also that I since there's just one utility of this kind so
far, and since we will have extras/contrib/adapters for things such as
BlockingConnection, I am content to drop proton.util and move URL to the
top level.
I would then consider a proton.logging group if there were enough things to
put there.
> Reference points:
>
> -
> https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;a=tree;f=examples/python
> - http://people.apache.org/~jross/proton-apidoc-draft-2/group__types.html
> -
> http://qpid.apache.org/releases/qpid-proton-0.8/protocol-engine/java/api/index.html
>
Gah, html frames got me. I meant to link to the list of proton-j types.
http://qpid.apache.org/releases/qpid-proton-0.8/protocol-engine/java/api/org/apache/qpid/proton/amqp/package-summary.html
Re: Proton API layout proposal
Posted by Justin Ross <ju...@gmail.com>.
On Wed, Feb 18, 2015 at 9:28 AM, Rafael Schloming <rh...@alum.mit.edu> wrote:
> A couple of comments in no particular order...
>
> The "Core user model and uitility classes" description could be
> misinterpreted on a quick read. Maybe go with "Core protocol model..." or
> something like that? (Really all these APIs are for users, so it seems
> weird to mention it.) Also the utility classes mentioned here sound like
> they overlap with the util package. What is that actually referring to?
>
Yes, good point. I'll go with protocol instead.
If I could, I'd make util go away. It's for low-level things (not Proton
extensions, but language library extensions) that we wish to make public.
The best example we currently have is URL.
> I like the proton.security addition, I think that makes sense as security
> is generally handled in one contained place in any application.
>
> I'm not so hot on the types subpackage. I don't think it actually buys us a
> lot in terms of uncluttering core since there are only a few very simple
> things in that category. It basically consists of about 5 or so pure
> constructors that extend builtin python types with extra information, e.g.
> timestamp which is just a subclass of long that gets encoded on the wire as
> a timestamp. It's implementation is literally just this[1], as you can see
> it adds very little surface area to the API. The other items in this
> category are similarly simple. Conceptually these types really are part of
> the core protocol model, and unlike in the case of security, if your app
> does need to deal with timestamps or unsigned numbers or whatever, it is
> pretty likely that it will need to do it all over the place which means
> that you basically always need to do two imports in order to get the full
> protocol model.
>
http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-types-v1.0-os.html#section-primitive-type-definitions
The number of AMQP primitive types is comparable to the number of endpoint
types. A given language may not need all the types, but it will likely
need more than half of them. Some of these will not find frequent use,
such as the decimal types.
I think it is a source of clutter (the implementation size is not a factor
here; instead it's the number of things in a namespace). The grouping has
the additional benefit of clarifying that these are all things of one sort.
Are there particular types that you think will find frequent use and should
be promoted to the "core" level? I too would like to consolidate the
imports, but I disagree about the scale of the clutter problem.
Gordon's python examples don't seem to import any types. For most higher
level languages, I would hope this would be the case.
Reference points:
-
https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;a=tree;f=examples/python
- http://people.apache.org/~jross/proton-apidoc-draft-2/group__types.html
-
http://qpid.apache.org/releases/qpid-proton-0.8/protocol-engine/java/api/index.html
Re: Proton API layout proposal
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Wed, Feb 18, 2015 at 4:18 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 02/17/2015 08:49 PM, Justin Ross wrote:
>
>> I've made a few changes to the api layout proposal:
>>
>>
>> https://cwiki.apache.org/confluence/display/qpid/
>> Proton+API+layout+proposal
>>
>> - Renamed proton.reactors to proton.reactor
>> - Added proton.security, to clean up the "core" space. This would be
>> home
>> to future additions such as message encryption as well.
>> - Went with proton.extras over proton.contrib. This was to avoid any
>> appearance that we are disowning the pieces there. Instead, they are
>> simply extensions outside the core Proton functionality.
>>
>
A couple of comments in no particular order...
The "Core user model and uitility classes" description could be
misinterpreted on a quick read. Maybe go with "Core protocol model..." or
something like that? (Really all these APIs are for users, so it seems
weird to mention it.) Also the utility classes mentioned here sound like
they overlap with the util package. What is that actually referring to?
I like the proton.security addition, I think that makes sense as security
is generally handled in one contained place in any application.
I'm not so hot on the types subpackage. I don't think it actually buys us a
lot in terms of uncluttering core since there are only a few very simple
things in that category. It basically consists of about 5 or so pure
constructors that extend builtin python types with extra information, e.g.
timestamp which is just a subclass of long that gets encoded on the wire as
a timestamp. It's implementation is literally just this[1], as you can see
it adds very little surface area to the API. The other items in this
category are similarly simple. Conceptually these types really are part of
the core protocol model, and unlike in the case of security, if your app
does need to deal with timestamps or unsigned numbers or whatever, it is
pretty likely that it will need to do it all over the place which means
that you basically always need to do two imports in order to get the full
protocol model.
[1]:
class timestamp(long):
def __repr__(self):
return "timestamp(%s)" % long.__repr__(self)
>
>> Gordon also put forward the notion of proton.adapters in lieu of
>> proton.extras. I like that as well, but I think it has a narrower sense,
>> and I was aiming to keep the buckets bigger and flatter.
>>
>
FWIW, the extras thing looks a bit odd to me. It kind of reads a bit like
"proton.bolt-ons", or "proton.stuff-we-couldn't-bother-to-name-properly". I
also don't think the "disown" thing is particularly meaningful (especially
not to end users). Contrib certainly isn't about disowning anything, it's
about layering. The stuff in contrib isn't allowed to depend on proton
internals, only on public APIs. Historically whether something is
"disowned" has nothing to do with what it is named, and much more to do
with whether or not it is maintained. We've deleted way more stuff in core
that got old and crufty and was unmaintained than we've deleted from
contrib. The contrib stuff, assuming it includes a reasonable test suite,
actually has a much better chance of just continuing to work with minimal
maintenance precisely because of the layering. That layering means that
when something breaks in the contrib tests it's because we introduced a bug
in a core API and so there is good incentive for us to fix it and make sure
the contrib stuff still works.
> To amplify a little, the point was that the two things currently in the
> utils module are ways of adapting the reactive, non-blocking, event-driven
> style to some other style (messenger is in my view a similar sort of
> thing). Though it is certainly more narrow, I think its also more helpful.
>
> The other aspect to these is that they aren't yet considered as
> fully-baked as the reactive core. I certainly don't object to them being in
> an extras namespace to denote that, until we are more comfortable we have
> the interfaces right. You could also however indicate that through
> documentation or some 'experimental' annotation.
I agree we don't necessarily need to signal bakedness in the package name,
I think it's good to think of it as an orthogonal dimension. I'm not quite
sure I follow the reasoning for proton.adapters though. That sounds to me
like somewhere we'd put stuff for integrations, e.g. maybe the tornado
stuff.
--Rafael
Re: Proton API layout proposal
Posted by Gordon Sim <gs...@redhat.com>.
On 02/17/2015 08:49 PM, Justin Ross wrote:
> I've made a few changes to the api layout proposal:
>
>
> https://cwiki.apache.org/confluence/display/qpid/Proton+API+layout+proposal
>
> - Renamed proton.reactors to proton.reactor
> - Added proton.security, to clean up the "core" space. This would be home
> to future additions such as message encryption as well.
> - Went with proton.extras over proton.contrib. This was to avoid any
> appearance that we are disowning the pieces there. Instead, they are
> simply extensions outside the core Proton functionality.
>
> Gordon also put forward the notion of proton.adapters in lieu of
> proton.extras. I like that as well, but I think it has a narrower sense,
> and I was aiming to keep the buckets bigger and flatter.
To amplify a little, the point was that the two things currently in the
utils module are ways of adapting the reactive, non-blocking,
event-driven style to some other style (messenger is in my view a
similar sort of thing). Though it is certainly more narrow, I think its
also more helpful.
The other aspect to these is that they aren't yet considered as
fully-baked as the reactive core. I certainly don't object to them being
in an extras namespace to denote that, until we are more comfortable we
have the interfaces right. You could also however indicate that through
documentation or some 'experimental' annotation.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton API layout proposal
Posted by Justin Ross <ju...@gmail.com>.
An update.
I've made a few changes to the api layout proposal:
https://cwiki.apache.org/confluence/display/qpid/Proton+API+layout+proposal
- Renamed proton.reactors to proton.reactor
- Added proton.security, to clean up the "core" space. This would be home
to future additions such as message encryption as well.
- Went with proton.extras over proton.contrib. This was to avoid any
appearance that we are disowning the pieces there. Instead, they are
simply extensions outside the core Proton functionality.
Gordon also put forward the notion of proton.adapters in lieu of
proton.extras. I like that as well, but I think it has a narrower sense,
and I was aiming to keep the buckets bigger and flatter.
The following is incomplete, but it at least previews the application of
these groupings:
http://people.apache.org/~jross/proton-apidoc-draft-2/modules.html
Once I fill all this in, I intend to use the main page tab to explain each
top-level group and tell the user what they need to focus on first.
Justin
On Wed, Feb 4, 2015 at 1:32 PM, Justin Ross <ju...@gmail.com> wrote:
>
>
> On Wed, Feb 4, 2015 at 1:00 PM, Rafael Schloming <rh...@alum.mit.edu> wrote:
>>
>> > Here's a first attempt:
>> >
>> > http://people.apache.org/~jross/proton-apidoc-draft/modules.html
>> >
>> > To summarize the delta from the current docs:
>> >
>> > - The groups are a flat array; this relates to the next point of
>> > difference
>> >
>> > - I used class-and-member relationships to achieve a grouping where it
>> > applies; I think it applies quite a bit, and the resulting api doc is
>> more
>> > natural to me
>> >
>>
>> I like this. I didn't know you could do this. What's the appropriate
>> markup
>> to make doxygen aware of the class/member relationship?
>>
>
> You put @class foo_t on the typedef and @memberof foo_t on the related
> functions. Once you've done that, the members fall in the group of the
> class without being explicitly marked up for it.
>
> http://people.apache.org/~jross/proton-apidoc-draft.diff
>
>
>> - I avoided bifurcating the API at the top level into engine and
>> > messenger; instead I tried to treat everything as the "Proton API"
>> >
>>
>> I like avoiding the bifurcation. The one thing I think isn't so great
>> about
>> this is when you click on the Endpoint group you get a huge listing of
>> stuff. It took me a couple of minutes to pick out the fact that the data
>> structure stuff basically linked to "class scoped" documentation. I think
>> in some ways the previous sub grouping scheme was a little better here
>> since you could navigate straight to Connection/Session/Link/whatever from
>> the modules page and just collapse those levels if you don't care about
>> the
>> sub groups. Is there a way to use the sub grouping along with the
>> class-and-member stuff to get the best of both worlds?
>>
>
> I agree that the picture under endpoint is confusing. I'll produce a few
> alternatives.
>
>
>> > - Where applicable, I used the groupings defined in the API layout
>> > proposal; note that I have pn_reactor_* in the Event group--that's
>> > something we talked about in our initial discussion of the API layout
>> >
>>
>> After actually using some of the APIs a bit, I'm wondering whether it
>> doesn't make sense to have the group called reactor and feature the
>> reactor
>> more heavily as an entry point. This is because the reactor is the first
>> thing you construct/interact with. In python you aren't really even aware
>> of events as a first class thing, they just get passed into handlers and
>> have useful stuff in them, but you never import or construct them
>> directly.
>> In C where you have types you do have to be a little bit more aware of
>> them, but it's really a secondary thing after you have set up your
>> reactor.
>>
>
> From what I've seen in the C api, a reactor group would make more sense.
>
>
Re: Proton API layout proposal
Posted by Justin Ross <ju...@gmail.com>.
On Wed, Feb 4, 2015 at 1:00 PM, Rafael Schloming <rh...@alum.mit.edu> wrote:
>
> > Here's a first attempt:
> >
> > http://people.apache.org/~jross/proton-apidoc-draft/modules.html
> >
> > To summarize the delta from the current docs:
> >
> > - The groups are a flat array; this relates to the next point of
> > difference
> >
> > - I used class-and-member relationships to achieve a grouping where it
> > applies; I think it applies quite a bit, and the resulting api doc is
> more
> > natural to me
> >
>
> I like this. I didn't know you could do this. What's the appropriate markup
> to make doxygen aware of the class/member relationship?
>
You put @class foo_t on the typedef and @memberof foo_t on the related
functions. Once you've done that, the members fall in the group of the
class without being explicitly marked up for it.
http://people.apache.org/~jross/proton-apidoc-draft.diff
> - I avoided bifurcating the API at the top level into engine and
> > messenger; instead I tried to treat everything as the "Proton API"
> >
>
> I like avoiding the bifurcation. The one thing I think isn't so great about
> this is when you click on the Endpoint group you get a huge listing of
> stuff. It took me a couple of minutes to pick out the fact that the data
> structure stuff basically linked to "class scoped" documentation. I think
> in some ways the previous sub grouping scheme was a little better here
> since you could navigate straight to Connection/Session/Link/whatever from
> the modules page and just collapse those levels if you don't care about the
> sub groups. Is there a way to use the sub grouping along with the
> class-and-member stuff to get the best of both worlds?
>
I agree that the picture under endpoint is confusing. I'll produce a few
alternatives.
> > - Where applicable, I used the groupings defined in the API layout
> > proposal; note that I have pn_reactor_* in the Event group--that's
> > something we talked about in our initial discussion of the API layout
> >
>
> After actually using some of the APIs a bit, I'm wondering whether it
> doesn't make sense to have the group called reactor and feature the reactor
> more heavily as an entry point. This is because the reactor is the first
> thing you construct/interact with. In python you aren't really even aware
> of events as a first class thing, they just get passed into handlers and
> have useful stuff in them, but you never import or construct them directly.
> In C where you have types you do have to be a little bit more aware of
> them, but it's really a secondary thing after you have set up your reactor.
>
>From what I've seen in the C api, a reactor group would make more sense.
Re: Proton API layout proposal
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Wed, Feb 4, 2015 at 10:16 AM, Justin Ross <ju...@gmail.com> wrote:
> On Thu, Jan 29, 2015 at 12:08 PM, Rafael Schloming <rh...@alum.mit.edu>
> wrote:
>
> > On Thu, Jan 29, 2015 at 9:28 AM, Justin Ross <ju...@gmail.com>
> > wrote:
> >
> > > This makes good sense on its own, but it doesn't help with our goal of
> > > naming and grouping together a "growable library of like things". In
> > other
> > > words, if we end up with many reactor implementations, it won't be very
> > > approachable to have them littered around as dissimilar peers to
> > reactor.h.
> > >
> >
> > I don't think it's really workable to think of C header files as simple
> > analogs to python packages. C header files need to first and formost
> > correspond to the C symbols they include. Breaking that correspondence
> does
> > a huge disservice to the usability of the API from C. Beyond that, the
> fact
> > that C doesn't actually have packages and has just one global namespace
> > really forces you to use flatter naming for your C symbols since
> everytime
> > you type a symbol, you need to type out its full global name. These two
> > things together pretty much guarantee that the C header files can't match
> > python package names unless you force everyone to type out the full
> python
> > package name as part of every C symbol.
> >
> >
> I take your point. As you suggested, I've got a scheme that uses doxygen
> to achieve the grouping goals I have in mind.
>
> In line with your expressed intent here, and having poured over the
> headers, I think there are a few instances where the symbol-to-header
> correspondence could be tighter: pn_data_*, pn_logger_t, pn_collector_*.
> In some instances, it seems like it would make sense to rename the symbol,
> and in others to add a corresponding header.
>
>
Yeah, some of that is just left over from early days. I know of a few other
examples from some dark corners.
There's also a general pattern that doesn't quite fit. With the attachments
stuff you can basically extend core objects with additional attributes. The
reactor does this, for example to attach scoped handlers to endpoints, as
well as to associate events with the reactor itself. The natural API for
this sort of thing is to simply have pn_event_reactor(event) ->
pn_reactor_t, however it would mess with the dependencies to actually
define this where we define the other pn_event_ symbols, so it is defined
in reactor.h.
> > If you want to improve approachability, doxygen has lots of tools for
> > defining groups and organizing/presenting otherwise flat APIs in a more
> > structured way. This is because the whole large flat C API is pretty
> > common, so generally it is organized with annotations in the doc tags
> that
> > doxygen then uses to create things that actually are more analogous to
> > python packages. This used to be set up, I'm not sure if it still is,
> but I
> > would suggest that this is probably the first place to look to start
> > improving accessibility.
> >
>
> Here's a first attempt:
>
> http://people.apache.org/~jross/proton-apidoc-draft/modules.html
>
> To summarize the delta from the current docs:
>
> - The groups are a flat array; this relates to the next point of
> difference
>
> - I used class-and-member relationships to achieve a grouping where it
> applies; I think it applies quite a bit, and the resulting api doc is more
> natural to me
>
I like this. I didn't know you could do this. What's the appropriate markup
to make doxygen aware of the class/member relationship?
- I avoided bifurcating the API at the top level into engine and
> messenger; instead I tried to treat everything as the "Proton API"
>
I like avoiding the bifurcation. The one thing I think isn't so great about
this is when you click on the Endpoint group you get a huge listing of
stuff. It took me a couple of minutes to pick out the fact that the data
structure stuff basically linked to "class scoped" documentation. I think
in some ways the previous sub grouping scheme was a little better here
since you could navigate straight to Connection/Session/Link/whatever from
the modules page and just collapse those levels if you don't care about the
sub groups. Is there a way to use the sub grouping along with the
class-and-member stuff to get the best of both worlds?
> - Where applicable, I used the groupings defined in the API layout
> proposal; note that I have pn_reactor_* in the Event group--that's
> something we talked about in our initial discussion of the API layout
>
After actually using some of the APIs a bit, I'm wondering whether it
doesn't make sense to have the group called reactor and feature the reactor
more heavily as an entry point. This is because the reactor is the first
thing you construct/interact with. In python you aren't really even aware
of events as a first class thing, they just get passed into handlers and
have useful stuff in them, but you never import or construct them directly.
In C where you have types you do have to be a little bit more aware of
them, but it's really a secondary thing after you have set up your reactor.
> I don't at all consider the api layout finalized, so if this effort points
> up some problems, let's adjust. The same of course goes for the C api doc.
Something to be aware of here is that my thinking on the need for multiple
reactors has changed a bit. I wrote a separate post about it since it got a
little lengthy to incude here, but the upshot is that the API layout
doesn't necessarily need to accommodate multiple completely distinct
reactors in the way that we previously had imagined, and that it may be
worth featuring the base reactor a little bit more prominently.
--Rafael
Re: Proton API layout proposal
Posted by Justin Ross <ju...@gmail.com>.
On Thu, Jan 29, 2015 at 12:08 PM, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Thu, Jan 29, 2015 at 9:28 AM, Justin Ross <ju...@gmail.com>
> wrote:
>
> > This makes good sense on its own, but it doesn't help with our goal of
> > naming and grouping together a "growable library of like things". In
> other
> > words, if we end up with many reactor implementations, it won't be very
> > approachable to have them littered around as dissimilar peers to
> reactor.h.
> >
>
> I don't think it's really workable to think of C header files as simple
> analogs to python packages. C header files need to first and formost
> correspond to the C symbols they include. Breaking that correspondence does
> a huge disservice to the usability of the API from C. Beyond that, the fact
> that C doesn't actually have packages and has just one global namespace
> really forces you to use flatter naming for your C symbols since everytime
> you type a symbol, you need to type out its full global name. These two
> things together pretty much guarantee that the C header files can't match
> python package names unless you force everyone to type out the full python
> package name as part of every C symbol.
>
>
I take your point. As you suggested, I've got a scheme that uses doxygen
to achieve the grouping goals I have in mind.
In line with your expressed intent here, and having poured over the
headers, I think there are a few instances where the symbol-to-header
correspondence could be tighter: pn_data_*, pn_logger_t, pn_collector_*.
In some instances, it seems like it would make sense to rename the symbol,
and in others to add a corresponding header.
> If you want to improve approachability, doxygen has lots of tools for
> defining groups and organizing/presenting otherwise flat APIs in a more
> structured way. This is because the whole large flat C API is pretty
> common, so generally it is organized with annotations in the doc tags that
> doxygen then uses to create things that actually are more analogous to
> python packages. This used to be set up, I'm not sure if it still is, but I
> would suggest that this is probably the first place to look to start
> improving accessibility.
>
Here's a first attempt:
http://people.apache.org/~jross/proton-apidoc-draft/modules.html
To summarize the delta from the current docs:
- The groups are a flat array; this relates to the next point of
difference
- I used class-and-member relationships to achieve a grouping where it
applies; I think it applies quite a bit, and the resulting api doc is more
natural to me
- I avoided bifurcating the API at the top level into engine and
messenger; instead I tried to treat everything as the "Proton API"
- Where applicable, I used the groupings defined in the API layout
proposal; note that I have pn_reactor_* in the Event group--that's
something we talked about in our initial discussion of the API layout
I don't at all consider the api layout finalized, so if this effort points
up some problems, let's adjust. The same of course goes for the C api doc.
Re: Proton API layout proposal
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Thu, Jan 29, 2015 at 9:28 AM, Justin Ross <ju...@gmail.com> wrote:
> This makes good sense on its own, but it doesn't help with our goal of
> naming and grouping together a "growable library of like things". In other
> words, if we end up with many reactor implementations, it won't be very
> approachable to have them littered around as dissimilar peers to reactor.h.
>
I don't think it's really workable to think of C header files as simple
analogs to python packages. C header files need to first and formost
correspond to the C symbols they include. Breaking that correspondence does
a huge disservice to the usability of the API from C. Beyond that, the fact
that C doesn't actually have packages and has just one global namespace
really forces you to use flatter naming for your C symbols since everytime
you type a symbol, you need to type out its full global name. These two
things together pretty much guarantee that the C header files can't match
python package names unless you force everyone to type out the full python
package name as part of every C symbol.
The one sortof exception to this is grouping header files which include a
bunch of commonly used stuff together, e.g. we have proton/engine.h
connection, session, link, delivery, etc, since they are all often used
together. It's not quite right to think of this as a python package either
though because it doesn't actually introduce any namespacing. It's really
just an alias for a bunch of common includes.
If you want to improve approachability, doxygen has lots of tools for
defining groups and organizing/presenting otherwise flat APIs in a more
structured way. This is because the whole large flat C API is pretty
common, so generally it is organized with annotations in the doc tags that
doxygen then uses to create things that actually are more analogous to
python packages. This used to be set up, I'm not sure if it still is, but I
would suggest that this is probably the first place to look to start
improving accessibility.
I assume this is why you have handlers.h: so you don't need to define a
> header file for each handler implementation. Down the line, we could add a
> reactors.h, but then we have reactor.h and reactors.h--not good.
>
The difference here is that you can't use multiple reactors simultaneously,
so it will never make sense to have a reactors.h. Whereas the handlers are
much more tightly coupled and they are expected to be intermingled with
each other. Also, these handlers aren't very configurable, so the extent of
the API is just one constructor. If we develop a more complicated handler,
e.g. pn_foo_t, with a lot of pn_foo_* methods, I would expect it to go into
proton/foo.h (and possibly be included from handlers.h for convenience.)
> We could address the concern through directory structure or perhaps through
> a naming convention: reactor.h with reactor_$variant.h. The latter pattern
> seems to be relatively common.
>
This would either break the mapping between header files and symbols, or
require unusably long symbol names.
> I find types.h strange against all of this. Are the endpoint types
> collected there in order to solve a dependency problem? It contains some
> but definitely not all of the non-amqp-datatype types.
>
Yes, it is to solve a dependency problem.
--Rafael
Re: Proton API layout proposal
Posted by Justin Ross <ju...@gmail.com>.
This makes good sense on its own, but it doesn't help with our goal of
naming and grouping together a "growable library of like things". In other
words, if we end up with many reactor implementations, it won't be very
approachable to have them littered around as dissimilar peers to reactor.h.
I assume this is why you have handlers.h: so you don't need to define a
header file for each handler implementation. Down the line, we could add a
reactors.h, but then we have reactor.h and reactors.h--not good.
We could address the concern through directory structure or perhaps through
a naming convention: reactor.h with reactor_$variant.h. The latter pattern
seems to be relatively common.
I find types.h strange against all of this. Are the endpoint types
collected there in order to solve a dependency problem? It contains some
but definitely not all of the non-amqp-datatype types.
http://qpid.apache.org/releases/qpid-proton-0.8/protocol-engine/c/api/types_8h.html
On Wed, Jan 28, 2015 at 5:25 PM, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Wed, Jan 28, 2015 at 3:36 PM, Justin Ross <ju...@gmail.com>
> wrote:
>
> >
> https://cwiki.apache.org/confluence/display/qpid/Proton+API+layout+proposal
> >
> > Please take a look. The intention:
> >
> > - Organize the API consistently across languages, allowing for
> deliberate
> > exceptions
> >
> > - Put more frequently used items closer to hand, and less frequently
> used
> > ones out of the way
> >
> > - Put like things together in a named group
> >
> > This scheme is now in use to some extent by the C and Python variants,
> but
> > there are some possibly non-semantic variations that I think are worth
> > ironing out:
> >
> > - Python is using utils, not util. I don't care whether we go util or
> > utils, but I'd very much like to avoid it being half one way and half the
> > other. If we decide utils, I'll update the wiki page.
> >
> > - C is using reactor, not reactors. Is the idea that it will have only
> > one reactor implementation?
> >
>
> No that's not the intention, although what is in reactor.h should probably
> be the basis for any other reactors.
>
> The header files in C actually have a much stronger correspondence to
> classes in python rather than than to packages. The C API uses header files
> to identify objects, e.g. foo.h generally has a typedef pn_foo_t and a
> bunch of "methods" named pn_foo_* for operating on pn_foo_t's. Sometimes
> there are two related types grouped together, e.g. proton/event.h contains
> pn_event_t and pn_collector_t since pn_event_t objects can't actually exist
> on their own without a collector.
>
> So some of the resemblance between C header files and python packages is
> actually less strong than you suggest. C is just flatter because it doesn't
> actually have packages.
>
> Hopefully this is clear by now, but the reason it's reactor.h and not
> reactors.h is because that is where pn_reactor_t and all it's related
> pn_reactor_* operations are defined. If we were to add another reactor,
> e.g. something based on libuv for example, I would expect that to go into
> "proton/uv.h" and probably inherit/reference/reuse a lot of what is in
> "proton/reactor.h".
>
> --Rafael
>
Re: Proton API layout proposal
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Wed, Jan 28, 2015 at 3:36 PM, Justin Ross <ju...@gmail.com> wrote:
> https://cwiki.apache.org/confluence/display/qpid/Proton+API+layout+proposal
>
> Please take a look. The intention:
>
> - Organize the API consistently across languages, allowing for deliberate
> exceptions
>
> - Put more frequently used items closer to hand, and less frequently used
> ones out of the way
>
> - Put like things together in a named group
>
> This scheme is now in use to some extent by the C and Python variants, but
> there are some possibly non-semantic variations that I think are worth
> ironing out:
>
> - Python is using utils, not util. I don't care whether we go util or
> utils, but I'd very much like to avoid it being half one way and half the
> other. If we decide utils, I'll update the wiki page.
>
> - C is using reactor, not reactors. Is the idea that it will have only
> one reactor implementation?
>
No that's not the intention, although what is in reactor.h should probably
be the basis for any other reactors.
The header files in C actually have a much stronger correspondence to
classes in python rather than than to packages. The C API uses header files
to identify objects, e.g. foo.h generally has a typedef pn_foo_t and a
bunch of "methods" named pn_foo_* for operating on pn_foo_t's. Sometimes
there are two related types grouped together, e.g. proton/event.h contains
pn_event_t and pn_collector_t since pn_event_t objects can't actually exist
on their own without a collector.
So some of the resemblance between C header files and python packages is
actually less strong than you suggest. C is just flatter because it doesn't
actually have packages.
Hopefully this is clear by now, but the reason it's reactor.h and not
reactors.h is because that is where pn_reactor_t and all it's related
pn_reactor_* operations are defined. If we were to add another reactor,
e.g. something based on libuv for example, I would expect that to go into
"proton/uv.h" and probably inherit/reference/reuse a lot of what is in
"proton/reactor.h".
--Rafael
Re: Proton API layout proposal
Posted by Justin Ross <ju...@gmail.com>.
Yes, quite a lot like contrib. I think that's a fine name. If no one else
objects, I'll add that to our api layout doc.
On Mon, Feb 2, 2015 at 4:20 PM, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Thu, Jan 29, 2015 at 9:56 AM, Justin Ross <ju...@gmail.com>
> wrote:
>
> > On Wed, Jan 28, 2015 at 3:36 PM, Justin Ross <ju...@gmail.com>
> > wrote:
> > >
> > > - Python is using utils, not util. I don't care whether we go util or
> > > utils, but I'd very much like to avoid it being half one way and half
> the
> > > other. If we decide utils, I'll update the wiki page.
> > >
> >
> > Distinct from the naming consistency question, util[s] has an ambiguity
> of
> > purpose that I think is important to resolve--particularly in Python,
> > because it's difficult to have clean dependencies if we don't.
> >
> > These are the two ways people tend use util:
> >
> > 1. Functions and classes for internal use by other parts of the proton
> > library; that is, handy stuff that your language library didn't give you
> > 2. Functions and classes for external use by consumers of proton
> >
> > PROTON-805 adds more of the latter to proton/utils.py, and it already had
> > BlockingConnection. Having both types of util in one module ends up
> making
> > imports a mess because it contains at once quite low-level things and
> > high-level things.
> >
> > Maybe we should have proton/extras.py for stuff like BlockingConnection
> and
> > SyncRequestResponseHandler and reserve utils.py for the internal utility
> > code.
> >
>
> Are you thinking of extras.py as being like contrib or something different?
> (FYI, we already have a contrib on the proton-j side of things, so having a
> proton/contrib.py would be consistent.)
>
> I definitely agree that PROTON-805 is making dubious use of the utils
> package, not only for the reasons you state, but also because the general
> request/response scenario should probably at some point be supported in a
> more core way, and that might result in a different API. I could be wrong
> since I haven't actually looked into how it works and it might be just
> fine, but I'd be inclined to put it into a more sandboxy area for the
> moment. Someplace where it's ok to leave it for a bit to provide backwards
> compatibility even if it's functionality is subsumed by other APIs.
>
> --Rafael
>
Re: Proton API layout proposal
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Thu, Jan 29, 2015 at 9:56 AM, Justin Ross <ju...@gmail.com> wrote:
> On Wed, Jan 28, 2015 at 3:36 PM, Justin Ross <ju...@gmail.com>
> wrote:
> >
> > - Python is using utils, not util. I don't care whether we go util or
> > utils, but I'd very much like to avoid it being half one way and half the
> > other. If we decide utils, I'll update the wiki page.
> >
>
> Distinct from the naming consistency question, util[s] has an ambiguity of
> purpose that I think is important to resolve--particularly in Python,
> because it's difficult to have clean dependencies if we don't.
>
> These are the two ways people tend use util:
>
> 1. Functions and classes for internal use by other parts of the proton
> library; that is, handy stuff that your language library didn't give you
> 2. Functions and classes for external use by consumers of proton
>
> PROTON-805 adds more of the latter to proton/utils.py, and it already had
> BlockingConnection. Having both types of util in one module ends up making
> imports a mess because it contains at once quite low-level things and
> high-level things.
>
> Maybe we should have proton/extras.py for stuff like BlockingConnection and
> SyncRequestResponseHandler and reserve utils.py for the internal utility
> code.
>
Are you thinking of extras.py as being like contrib or something different?
(FYI, we already have a contrib on the proton-j side of things, so having a
proton/contrib.py would be consistent.)
I definitely agree that PROTON-805 is making dubious use of the utils
package, not only for the reasons you state, but also because the general
request/response scenario should probably at some point be supported in a
more core way, and that might result in a different API. I could be wrong
since I haven't actually looked into how it works and it might be just
fine, but I'd be inclined to put it into a more sandboxy area for the
moment. Someplace where it's ok to leave it for a bit to provide backwards
compatibility even if it's functionality is subsumed by other APIs.
--Rafael
Re: Proton API layout proposal
Posted by Justin Ross <ju...@gmail.com>.
On Thu, Jan 29, 2015 at 11:35 AM, Andrew Stitcher <as...@redhat.com>
wrote:
> On Thu, 2015-01-29 at 09:56 -0500, Justin Ross wrote:
> > ...
> > PROTON-805 adds more of the latter to proton/utils.py, and it already had
> > BlockingConnection. Having both types of util in one module ends up
> making
> > imports a mess because it contains at once quite low-level things and
> > high-level things.
>
> If some of the things in the module are for internal library only use
> then they should not be exported and to some extent the namespace is
> irrelevant. I am aware that python does not do this terribly well, and
> that may be a reason in python (and other languages deficient in access
> control) to have an "internal" namespace.
>
Sorry, I should have said *primarily* for internal use. It's more about
their place in the dependency stack. They are language-library extensions,
not proton-library extensions. I think there are (or might be) legitimate
cases where these may be public, such as for a Url class.
> Maybe we should have proton/extras.py for stuff like BlockingConnection
> and
> > SyncRequestResponseHandler and reserve utils.py for the internal utility
>
> For internal utilities I strongly suggest a name that says - do not use
> this internal library use only.
>
I very much agree that we should do that where we can. Indeed, there's
some work to do in the python code to avoid leaking symbols we don't intend
for wide use. In python, it's easy to do inadvertently.
Re: Proton API layout proposal
Posted by Andrew Stitcher <as...@redhat.com>.
On Thu, 2015-01-29 at 09:56 -0500, Justin Ross wrote:
> ...
> PROTON-805 adds more of the latter to proton/utils.py, and it already had
> BlockingConnection. Having both types of util in one module ends up making
> imports a mess because it contains at once quite low-level things and
> high-level things.
If some of the things in the module are for internal library only use
then they should not be exported and to some extent the namespace is
irrelevant. I am aware that python does not do this terribly well, and
that may be a reason in python (and other languages deficient in access
control) to have an "internal" namespace.
>
> Maybe we should have proton/extras.py for stuff like BlockingConnection and
> SyncRequestResponseHandler and reserve utils.py for the internal utility
For internal utilities I strongly suggest a name that says - do not use
this internal library use only.
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton API layout proposal
Posted by Justin Ross <ju...@gmail.com>.
On Wed, Jan 28, 2015 at 3:36 PM, Justin Ross <ju...@gmail.com> wrote:
>
> - Python is using utils, not util. I don't care whether we go util or
> utils, but I'd very much like to avoid it being half one way and half the
> other. If we decide utils, I'll update the wiki page.
>
Distinct from the naming consistency question, util[s] has an ambiguity of
purpose that I think is important to resolve--particularly in Python,
because it's difficult to have clean dependencies if we don't.
These are the two ways people tend use util:
1. Functions and classes for internal use by other parts of the proton
library; that is, handy stuff that your language library didn't give you
2. Functions and classes for external use by consumers of proton
PROTON-805 adds more of the latter to proton/utils.py, and it already had
BlockingConnection. Having both types of util in one module ends up making
imports a mess because it contains at once quite low-level things and
high-level things.
Maybe we should have proton/extras.py for stuff like BlockingConnection and
SyncRequestResponseHandler and reserve utils.py for the internal utility
code.