on how to use the qpid java client

[Gordon Sim <gs...@redhat.com>]

rajith@apache.org wrote:
> Author: rajith
> Date: Fri Jun  1 13:40:34 2007
> New Revision: 543602
> 
> URL: http://svn.apache.org/viewvc?view=rev&rev=543602
> Log:
> more enchancements for the Qpid java client. Also I have checked in a sample client(QpidTestClient) on how to use the qpid java client.

Rajith,

Great to see this coming together! The sample was perfect for getting a 
quick overview of how the API is to work. It will also be very useful in 
focusing the discussion on more uniformity across the qpid clients.

I've attached some comments on that sample. I have to confess though 
that I didn't investigate the APIs directly so some of these comments 
may be based on ignorance of what is already there. A lot of them are 
also probably based on prejudices acquired in connection with the c++ or 
python clients!

--Gordon.

RE: on how to use the qpid java client

[Tomas Restrepo <to...@devdeo.com>]

Rupert,


> Might it be an idea to hold/complete the discussion on uniformity
> accross
> clients, before solidifying the choices made into Java code?
> 
> It seems to me from the .Net API discussion that the following are the
> most
> favoured choices:
> 
> WCF -> Qpid AMQ API -> framing
> JMS -> Qpid AMQ API -> framing
> 
> I had some questions about possible efficiency issues around too much
> layering, but as the Qpid API maps so closely onto the AMQP protocol
> that
> the middle bit is perhaps little more than some convenience methods,
> that
> you might write anyway, to build and dispatch frames to/from the event
> framework and framing layer that needs to be there regardless of the
> API
> choice.
> 
> Do we need/want/have the will/time to expose a common Qpid AMQ API
> accross
> all clients? If we want uniformity, and some choices and changes are
> currently being made as to what the .Net API will look like, it would
> be
> prudent to reach a concensus first, or else we will just end up with
> two
> more non-uniform APIs.

I agree this is a valid question, but at least from my point of view,
uniformity of the API isn't such a big deal, for this kind of project in
particular. That said, if the API were to be that AMQP-compatible API, which
is certainly fairly low level, then by that virtue alone the API would be
fairly similar across implementations, simply because it would resemble the
AMQP class/method model fairly closely. I believe we'd likely still have
some differences for things async operations (which definitely have
different conventions across languages), but that wouldn't be that bad.

Personally, I think this is a good thing, and I'd vote +1 on creating this
lower level API. Seems like 0.10/0.11 might be a good time to do it given
the changes in sessions/channel and framing, but what do I know :)

Also, just for the record, from the .NET client side: Sure, it would be nice
to have a WCF channel for Qpid; in fact, it's something I've in mind for a
long time, and even made some initial implementation. However, having a
lower-level, alternative API would still be a good thing, for several
reasons:

1- Not all people will want to use WCF, for example, we might want to
provide native BizTalk adapters as an alternative.
2- WCF limits what the API can do, simply because the surface it exposes to
the application/service is limited to the messaging operations (i.e.
send/receive). There's really no clean way of exposing the other AMQP API's
to the application/service to do things like create a queue on demand, or
create an exchange.

In the initial WCF channel implementation I did, I worked around this by
using configuration properties that provide support for common operating
models. For example, it might be common for a message publisher to declare
the exchange it will publish messages to, so it could set a property on the
binding telling it to declare the exchange (alongside with the exchange name
and class properties) when the client channel is opened. Now, this is a good
compromise, I think, but it still is a compromise.

 
> Also, what are we to make of the new Java client? I'm a bit unclear as
> to
> why it is being written in a completely seperate module as a clean
> re-implementation. 

I can't really comment on this, but I do have an opinion regarding it. I've
become very familiar with the Java client code (as well as the .NET client
code) over the past few months, and in my very humble opinion, a lot of the
code is a mess. Not only is it complex, it is also hard to test and it has a
number of poor practices that make it even more complex: Classes too big
(just look at the connection or channel/session classes), very tight
coupling between classes at different layers and an arcane threading model
(i.e. MINA in the Java code, and unfortunately the original .NET code got
stuck with the same model, even though it doesn't have the MINA
restrictions). This not only means the code is harder to maintain (again,
imho), it is also more fragile and inefficient (and the multitude of threads
it creates here and there to work around the base threading model makes it
"steal" valuable resources from the application/service using the client).

Personally, if it were up to me, and we agreed on creating the new
lower-level API, at least for the .NET client I think the right choice would
be to rewrite a large part of the client code base. A lot of this will be
necessary anyway with the changes in 0.10/11, anyway. That said, the lowest
levels of the stack work pretty well, imho (i.e. things the model for the
lower level framing layer and coding/decoding and the transport layer).

Just to put it in context: I was pretty surprised (and not in a good way) to
realize that I was able to follow the C++ client code base much easier than
the java (or .net, for that matter) code base :)


Tomas Restrepo
http://www.winterdom.com/weblog/

Re: on how to use the qpid java client

[Jonathan Robie <jo...@redhat.com>]

Rupert Smith wrote:
> Do we need/want/have the will/time to expose a common Qpid AMQ API 
> accross
> all clients? If we want uniformity, and some choices and changes are
> currently being made as to what the .Net API will look like, it would be
> prudent to reach a concensus first, or else we will just end up with two
> more non-uniform APIs.

I absolutely agree that the goal should be to have consistent APIs 
across languages, including Java. Obviously, we also need really good 
support for JMS, in addition to the native AMQ API.

> Also, I could probably take the
> AMQSession class, chop into two pieces, and I would end up with a JMS API
> and an AMQP API more cleanly separated. I feel there is value in adapting
> the old into the new, to re-use existing code as much as possible, to 
> make
> sure some of the 'clever bits' don't get forgotten, and also to ease
> refactoring of code, tests for example, that have been written to the AMQ
> API rather than JMS, to be refactored along at the same time.

Makes sense to me - the fewer code bases to deal with the better ....

> Do note that I'm not saying that the design of the new client API is 
> wrong.
> I had a quick look at these examples, and it all seems right enough to 
> me.

I agree.

Jonathan

Re: on how to use the qpid java client

[Gordon Sim <gs...@redhat.com>]

Rupert Smith wrote:
> Might it be an idea to hold/complete the discussion on uniformity accross
> clients, before solidifying the choices made into Java code?

I don't believe the API used in Rajith's example is 'solid' in the sense 
of 'unchangeable'; thats why I aired my view on aspects of it.

I'd suggest anyone else with a view on what a 'uniform' API or a 
particular client's API should look like should share that.

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

Alan,

We already have a low level AMQP protocol API.
It's exactly a 1-1 map of the protocol level classes.
Similar to Python u can do message.ok , message.transfer ,
connection.secure..etc

Regards,

Rajith.

On 6/6/07, Alan Conway <ac...@redhat.com> wrote:
>
> On Mon, 2007-06-04 at 15:19 +0100, Rupert Smith wrote:
> > On 04/06/07, Tomas Restrepo <to...@devdeo.com> wrote:
> >
> > > I agree this is a valid question, but at least from my point of view,
> > > uniformity of the API isn't such a big deal, for this kind of project
> in
> > > particular. That said, if the API were to be that AMQP-compatible API,
> > > which
> > > is certainly fairly low level, then by that virtue alone the API would
> be
> > > fairly similar across implementations, simply because it would
> resemble
> > > the
> > > AMQP class/method model fairly closely. I believe we'd likely still
> have
> > > some differences for things async operations (which definitely have
> > > different conventions across languages), but that wouldn't be that
> bad.
> > >
> >
> >
> > I can be brought round to this point of view. As you rightly point out
> the
> > API is going to look like the protocol anyway. Also Robert Greigs post
> about
> > developers using the appropriate API for their language and that being
> the
> > one to implement, clarifies what is more important.
> >
> > Rupert
>
> +1 to this: establishing a similar "raw AMQP" API layer in each project
> based on the AMQP XML and building on top of that. I say "similar" not
> "uniform": the XML is the rosetta stone, each language should map the
> XML in the way best suited to that language. However it should be
> possible (although not necessarily promoted to users) to write a client
> on _only_ this API. Higher level APIs (JMS, the current C++ client etc.)
> would be layered on top.
>
> This solves a lot of problems: XML provides common terms that will be
> used in class & method names, this layer can (should) be code generated
> as far as possible so it will automatically extend to cover new protocol
> features, and it will be easier for developers to work across projects -
> they know the concepts, just need to learn the local mapping rules.
>
> This API is low level and tied to protocol changes, so it's not suitable
> for some customers: JMS, NMS and other high level APIs should be based
> on what users in that language are used to.
>
> Python is the best (only?) example so far of a clean "raw AMQP" client.
> I've been wanting to do a consistent "raw AMQP" underpinning for C++ for
> some time now. Is this something to consider for the current Java client
> work?
>
> Cheers,
> Alan.
>
>
>
>

Re: on how to use the qpid java client

[Alan Conway <ac...@redhat.com>]

On Mon, 2007-06-04 at 15:19 +0100, Rupert Smith wrote:
> On 04/06/07, Tomas Restrepo <to...@devdeo.com> wrote:
> 
> > I agree this is a valid question, but at least from my point of view,
> > uniformity of the API isn't such a big deal, for this kind of project in
> > particular. That said, if the API were to be that AMQP-compatible API,
> > which
> > is certainly fairly low level, then by that virtue alone the API would be
> > fairly similar across implementations, simply because it would resemble
> > the
> > AMQP class/method model fairly closely. I believe we'd likely still have
> > some differences for things async operations (which definitely have
> > different conventions across languages), but that wouldn't be that bad.
> >
> 
> 
> I can be brought round to this point of view. As you rightly point out the
> API is going to look like the protocol anyway. Also Robert Greigs post about
> developers using the appropriate API for their language and that being the
> one to implement, clarifies what is more important.
> 
> Rupert

+1 to this: establishing a similar "raw AMQP" API layer in each project
based on the AMQP XML and building on top of that. I say "similar" not
"uniform": the XML is the rosetta stone, each language should map the
XML in the way best suited to that language. However it should be
possible (although not necessarily promoted to users) to write a client
on _only_ this API. Higher level APIs (JMS, the current C++ client etc.)
would be layered on top.

This solves a lot of problems: XML provides common terms that will be
used in class & method names, this layer can (should) be code generated
as far as possible so it will automatically extend to cover new protocol
features, and it will be easier for developers to work across projects -
they know the concepts, just need to learn the local mapping rules.

This API is low level and tied to protocol changes, so it's not suitable
for some customers: JMS, NMS and other high level APIs should be based
on what users in that language are used to.

Python is the best (only?) example so far of a clean "raw AMQP" client.
I've been wanting to do a consistent "raw AMQP" underpinning for C++ for
some time now. Is this something to consider for the current Java client
work?

Cheers,
Alan.

Re: on how to use the qpid java client

[Rupert Smith <ru...@googlemail.com>]

On 04/06/07, Tomas Restrepo <to...@devdeo.com> wrote:

> I agree this is a valid question, but at least from my point of view,
> uniformity of the API isn't such a big deal, for this kind of project in
> particular. That said, if the API were to be that AMQP-compatible API,
> which
> is certainly fairly low level, then by that virtue alone the API would be
> fairly similar across implementations, simply because it would resemble
> the
> AMQP class/method model fairly closely. I believe we'd likely still have
> some differences for things async operations (which definitely have
> different conventions across languages), but that wouldn't be that bad.
>


I can be brought round to this point of view. As you rightly point out the
API is going to look like the protocol anyway. Also Robert Greigs post about
developers using the appropriate API for their language and that being the
one to implement, clarifies what is more important.

Rupert

Re: on how to use the qpid java client

[Rupert Smith <ru...@googlemail.com>]

On 04/06/07, Tomas Restrepo <to...@devdeo.com> wrote:

> I can't really comment on this, but I do have an opinion regarding it.
> I've
> become very familiar with the Java client code (as well as the .NET client
> code) over the past few months, and in my very humble opinion, a lot of
> the
> code is a mess. Not only is it complex, it is also hard to test and it has
> a
> number of poor practices that make it even more complex: Classes too big
> (just look at the connection or channel/session classes), very tight
> coupling between classes at different layers and an arcane threading model
> (i.e. MINA in the Java code, and unfortunately the original .NET code got
> stuck with the same model, even though it doesn't have the MINA
> restrictions). This not only means the code is harder to maintain (again,
> imho), it is also more fragile and inefficient (and the multitude of
> threads
> it creates here and there to work around the base threading model makes it
>
> "steal" valuable resources from the application/service using the client).
>

Yup, its a mess. It has grown to be a bit of a mess as solutions to various
problems have crept in. What I'm saying is that it is important to
understand what those problems are, because a clean implementation may well
forget about them, only to have to re-implement them some other way. There's
some tricky stuff going on that is not well documented, and I think it is
worth perservering with it, to get a cleaner solution out of it. I'd just
prefer to put a cleaner structure back into whats already there, rather than
throw a lot of work out and start again.

Rupert

Re: on how to use the qpid java client

[Rupert Smith <ru...@googlemail.com>]

On 04/06/07, Gordon Sim <gs...@redhat.com> wrote:

> Great to see this coming together! The sample was perfect for getting a
> quick overview of how the API is to work. It will also be very useful in
> focusing the discussion on more uniformity across the qpid clients.
>
>
Might it be an idea to hold/complete the discussion on uniformity accross
clients, before solidifying the choices made into Java code?

It seems to me from the .Net API discussion that the following are the most
favoured choices:

WCF -> Qpid AMQ API -> framing
JMS -> Qpid AMQ API -> framing

I had some questions about possible efficiency issues around too much
layering, but as the Qpid API maps so closely onto the AMQP protocol that
the middle bit is perhaps little more than some convenience methods, that
you might write anyway, to build and dispatch frames to/from the event
framework and framing layer that needs to be there regardless of the API
choice.

Do we need/want/have the will/time to expose a common Qpid AMQ API accross
all clients? If we want uniformity, and some choices and changes are
currently being made as to what the .Net API will look like, it would be
prudent to reach a concensus first, or else we will just end up with two
more non-uniform APIs.

Also, what are we to make of the new Java client? I'm a bit unclear as to
why it is being written in a completely seperate module as a clean
re-implementation. Is it currently an experiment to try out a slightly
different API for AMQP? Is it because the existing code is so horrible that
it was easier to start from scratch? I've just spent a bit of time cleaning
up some stuff around fail-over (for M2), so I know there is some tricky
stuff going on in there, starting from scratch some of the problems might be
have to be solved a second time, and what is the guarantee that a clean
implementation will end up with a better solution? Granted the way fail-over
works will change for the new protocol, but I think there is a lot to be
learned from the existing solution. Also, I could probably take the
AMQSession class, chop into two pieces, and I would end up with a JMS API
and an AMQP API more cleanly seperated. I feel there is value in adapting
the old into the new, to re-use existing code as much as possible, to make
sure some of the 'clever bits' don't get forgotten, and also to ease
refactoring of code, tests for example, that have been written to the AMQ
API rather than JMS, to be refactored along at the same time.

I'm just wondering what will happen to existing code written against the old
client. It will use JMS so hopefully no impact, but there are some specific
properties that exisiting code is using, like the amqp strict flag, names of
properties passed to the JNDI and so on. I can foresee a situation where we
have existing customers using the 'old' client and new ones using the 'new'
client and us having to support both for fear of breaking existing apps. I'm
keen to avoid this.

Do note that I'm not saying that the design of the new client API is wrong.
I had a quick look at these examples, and it all seems right enough to me.

Does the new client retrotranslate to 1.4? I notice fairly heavy use
of 1.5features?

Yet another exception hierarchy being introduced in the code base.
AMQPException with 7 constuctors... Do we really want to end up with:

newClientMethod() throws AMQPException
{
try {
 call to framing layer;
}
catch (AMQException e)
{
throw new AMQPException("went wrong!", e);
}
}

jmsImplementationMethod() throws JMSException
{
try {
 newclientmethod();
}
catch(AMQPException e)
{
throw new JMSException("went wrong!", e);
}
}

Rupert

Re: Baby Ben!

[Marnie McCormack <ma...@googlemail.com>]

Thanks for the good wishes everyone - and yes, qpid-dev seems a lovely oasis
of calm and quiet compared to our house !

I'll send out a photo (not to the public list) soon.

Take Care,
Marnie


On 6/5/07, Rajith Attapattu <ra...@gmail.com> wrote:
>
> +1 for all suggestions :)
>
> Marnie congrats and wish u and the family the very best.
>
> Regards,
>
> Rajith
>
> On 6/5/07, John O'Hara <jo...@gmail.com> wrote:
> >
> > Congratulations!
> > Agree with Rupert - how are you finding time to spend with us lot?
> >
> > A picture would be great, but probably not to the public list!
> >
> > All the very best
> > John
> >
> > On 05/06/07, Rupert Smith <ru...@googlemail.com> wrote:
> > >
> > > Congratulations Marnie, I'm surprised you still have time to mail the
> > > dev-list. Maybe its starting to look like a relaxing alternative.
> Aren't
> > > you
> > > going to spam the list with a photo?
> > >
> > > Rupert
> > >
> > > On 05/06/07, Kim van der Riet <ki...@redhat.com> wrote:
> > > >
> > > > On Tue, 2007-06-05 at 13:19 +0100, Gordon Sim wrote:
> > > > > Marnie McCormack wrote:
> > > > > > ps Baby Ben arrived on the 28th of May, weighing in at 4.84Kg(10
> > > lbs
> > > > 14
> > > > > > oz). All well but a bit hectic :-)
> > > > >
> > > > > Congratulations!
> > > > +1
> > > >
> > > > Kim
> > > >
> > > >
> > >
> >
>

Re: Baby Ben!

[Rajith Attapattu <ra...@gmail.com>]

+1 for all suggestions :)

Marnie congrats and wish u and the family the very best.

Regards,

Rajith

On 6/5/07, John O'Hara <jo...@gmail.com> wrote:
>
> Congratulations!
> Agree with Rupert - how are you finding time to spend with us lot?
>
> A picture would be great, but probably not to the public list!
>
> All the very best
> John
>
> On 05/06/07, Rupert Smith <ru...@googlemail.com> wrote:
> >
> > Congratulations Marnie, I'm surprised you still have time to mail the
> > dev-list. Maybe its starting to look like a relaxing alternative. Aren't
> > you
> > going to spam the list with a photo?
> >
> > Rupert
> >
> > On 05/06/07, Kim van der Riet <ki...@redhat.com> wrote:
> > >
> > > On Tue, 2007-06-05 at 13:19 +0100, Gordon Sim wrote:
> > > > Marnie McCormack wrote:
> > > > > ps Baby Ben arrived on the 28th of May, weighing in at 4.84Kg (10
> > lbs
> > > 14
> > > > > oz). All well but a bit hectic :-)
> > > >
> > > > Congratulations!
> > > +1
> > >
> > > Kim
> > >
> > >
> >
>

Re: Baby Ben!

[John O'Hara <jo...@gmail.com>]

Congratulations!
Agree with Rupert - how are you finding time to spend with us lot?

A picture would be great, but probably not to the public list!

All the very best
John

On 05/06/07, Rupert Smith <ru...@googlemail.com> wrote:
>
> Congratulations Marnie, I'm surprised you still have time to mail the
> dev-list. Maybe its starting to look like a relaxing alternative. Aren't
> you
> going to spam the list with a photo?
>
> Rupert
>
> On 05/06/07, Kim van der Riet <ki...@redhat.com> wrote:
> >
> > On Tue, 2007-06-05 at 13:19 +0100, Gordon Sim wrote:
> > > Marnie McCormack wrote:
> > > > ps Baby Ben arrived on the 28th of May, weighing in at 4.84Kg (10
> lbs
> > 14
> > > > oz). All well but a bit hectic :-)
> > >
> > > Congratulations!
> > +1
> >
> > Kim
> >
> >
>

Re: Baby Ben!

[Rupert Smith <ru...@googlemail.com>]

Congratulations Marnie, I'm surprised you still have time to mail the
dev-list. Maybe its starting to look like a relaxing alternative. Aren't you
going to spam the list with a photo?

Rupert

On 05/06/07, Kim van der Riet <ki...@redhat.com> wrote:
>
> On Tue, 2007-06-05 at 13:19 +0100, Gordon Sim wrote:
> > Marnie McCormack wrote:
> > > ps Baby Ben arrived on the 28th of May, weighing in at 4.84Kg (10 lbs
> 14
> > > oz). All well but a bit hectic :-)
> >
> > Congratulations!
> +1
>
> Kim
>
>

Re: Baby Ben!

[Kim van der Riet <ki...@redhat.com>]

On Tue, 2007-06-05 at 13:19 +0100, Gordon Sim wrote:
> Marnie McCormack wrote:
> > ps Baby Ben arrived on the 28th of May, weighing in at 4.84Kg (10 lbs 14
> > oz). All well but a bit hectic :-)
> 
> Congratulations!
+1

Kim

Re: Baby Ben!

[Gordon Sim <gs...@redhat.com>]

Marnie McCormack wrote:
> ps Baby Ben arrived on the 28th of May, weighing in at 4.84Kg (10 lbs 14
> oz). All well but a bit hectic :-)

Congratulations!

Re: on how to use the qpid java client

[Marnie McCormack <ma...@googlemail.com>]

Hi All,

I haven't had time to look at the re-work Rajith has committed just yet, but
wanted to contribute to the discussion about standard APIs across
implementations ...

I agree that we don't need to force a 'one size fits all' across the
different languages, and that this would probably be fairly horrid to do.
However, we do need to clarify and probably make consistent the assumptions
that the most used methods have under the covers.

For example, using a no-arg constructor to create a channel/session etc will
mean that the channel is/is not started or is/is not ready to use etc etc.
I'm a bit patchy but this kind of thing gets more complex because of the
various JMS assumptions behind the java client code.

We've had what could loosely be described as inter-op problems in the past
to do with the way that different client encode data (type mismatches etc).
I know some of this was the java client having to pre-empt the protocol work
on type encoding, but I think the issue still stands regardless.

I hope this make sense - though I'm not sure what a good & painless way to
do this is - perhaps the face-to-face is a good opportunity to discuss the
set API calls most likely to suffer from this type of inconsistency and
agree a simple doc-ing of what each method will do in all clients ?

Hth !

Regards,
Marnie

ps Baby Ben arrived on the 28th of May, weighing in at 4.84Kg (10 lbs 14
oz). All well but a bit hectic :-)


On 6/5/07, Rupert Smith <ru...@googlemail.com> wrote:
>
> On 04/06/07, Rajith Attapattu <ra...@gmail.com> wrote:
> >
> > > Rather than having to remember the format of this URL scheme, I'd
> > > prefer (overloaded) method arguments for the constitutent parts. E.g.
> > > con.connect("localhost", 5672, "test", "guest", "guest"). Or perhaps a
> > > properties object with some defined keys (i.e. similar to how JDBC
> > > connections are often configured).
>
>
> The existing AMQConnection class provides such a method...
>
>
> > > or (probably even more controversially!):
> > >
> > > session.exchangeDeclare(false, false ...);
> > > session.queueDeclare(false, false, ...);
> > > session.queueBind(QpidConstants.DIRECT_EXCHANGE_NAME, ...);
>
>
> The existing AMQSession does it this way... Length of the class is not the
> problem, undocumented code, lack of clean encapsulation of responsibilies,
> several classes all tightly bound together is. This can all be solved with
> my refactoring axe! The session implementation is always going to be long,
> just by the number of methods defined in javax.jms.Session, that by itself
> does not make it bad.
>
> Gordon, see my above comment. Creating messages can be hidden behind
> > factories.
>
>
> Like the existing set of message factories?
>
> I would like to ask my question again, about what is the reason for
> writing
> a new client from scratch? and what is the plan for integrating it into
> the
> existing code? Is it just an experiment with a new design that will be
> used
> to influence the existing client, or is the intention that the existing
> client is to be completely scrapped and replaced?
>
> I don't want to try and defend the existing code too heavily, as already
> pointed out, it is ugly in places but it seems to me that there is
> re-invention going on when there is already an adequate solution for the
> problem. Rather than have an API that I can call in a slightly different
> way
> to what is there already, I think I'd prefer some javadoc to explain what
> all the existing parameters are for. What concerns me about throwing away
> the existing solution is the time spend re-solving the same problems
> again,
> and the danger of throwing out bug fixes that have been put in as feedback
> from customers required them.
>
> The existing client is maturing. Probably it was coded for the success
> pathway first, where things could go wrong the original authors just threw
> an AMQException or JMSException and forgot about it. Then some of these
> failure pathways have been examined more closely, in response to testing
> and
> bug reports and time spent thinking. There are still ways this client can
> fail in unexpected ways, but it is getting there, and I feel more than
> anything it is in need of completion, by closing off these existing
> failures, and fully testing the expected error pathways as well as the
> success ones (even better, by simply removing some of the error pathways
> that cannot occurr), and by writing the javadoc. Starting from scratch
> means
> undergoing this maturation process again.
>
> One of the disadvantages of implementing a new client from scratch in a
> completely new package is that existing code, tests for example, will not
> be
> refactored along at the same time. If you refactor, using say intellij,
> you
> will keep this code up-to-date as you go. For example, if I safe delete a
> method, I will have to go through the tests and refactor to the new API to
> get things to compile again, then re-run the tests keeping the test suite
> up-to-date with developments.
>
> Going off on a tangent, here's a simple scheme I have used succesfully in
> the past, for regression testing old bugs as code evolves - bug report
> goes
> in Jira Qpid-203, so you write a test called testQpid203 to expose the
> bug,
> so long as the test is passing the bug remains solved.
>
> Here is a list of things that are buried somewhere in the existing code,
> that a complete re-implementation might forget about, or have to re-invent
> in a very similar fashion:
>
> Hashed password implementation for SASL.
> The ability to turn off prefetch on new connections to work with MULE
> (MULEs
> fault and a pitty we have to have stuff like this in our code, but thats
> the
> real world creeping in).
> ConnectionListener to listen for failovers (unfortunately we have an
> existing customer using this non-JMS feature... their risk to bear for
> being
> non-JMS, I suppose).
> Existing system properties such as amq strict flag.
> Synchronous request/response that can be interrupted by fail-over. The way
> fail-over works will change for the new protocol, because it supports it
> better, but a similar solution involving interruptable tasks and some sort
> of retry will inevitably be invented.
>
> This is just stuff that I know of, I haven't fully examined the existing
> code, so there are probably other things too.
>
> The danger is leaving existing customers behind on an old branch, and not
> being able to bring them easily onto the new branch, for fear of
> re-introducing old bugs, or incompatible API changes. I did post about
> APIs
> months ago, to make the point that an API needs to be decided early and
> preferably set in stone from then on (things are a bit different for the
> .Net because there are less people using it in the wild yet).
> Unfortunately,
> you need to give some carefull consideration before changing a public API
> method, and the nice way to do it is to mark the old one @deprecated for a
> while, before removing it.
>
> Rupert
>

Re: on how to use the qpid java client

[Rupert Smith <ru...@googlemail.com>]

On 06/06/07, Rajith Attapattu <ra...@gmail.com> wrote:

> As for the flexible part considrer the following example. If I want to
> send
> the same message to several destinations (different exchanges/routing
> key),
> I can use the low level client to do  a "transfer by reference" where I
> send
> the content once  followed by a several transfer methods to send the
> different message headers.
>

Wouldn't you just use a topic or fanout? Or in some more esoteric cases set
up forwarding exchanges? Then you'd do mulitple bindings at the start but
only one send per message, instead of multiple transfer methods?

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 06/06/07, Rajith Attapattu <ra...@gmail.com> wrote:

> [RA] Sorry I don't buy this argument at all. Every project needs to make
> hard decisions and live with it.
> You know this more than me. IMHO It's better to have choices. People always
> feel they could have done it better after a while.
> This dilemma is every where. Should I use EJB3 or Spring/Hibernate ? should
> use JPA or not? Should I use log4j or not?

Every project does need to make decisions. But what I am saying is
that the cost of change is high. Given that, people may not want to
choose a less powerful API even though at the point they are making a
decision they cannot see an immediate need for a particular feature.

> These are questions almost everybody runs into. Before evaluating what
> options to use they need to evaluate their long term goals of the project.
> If they think they might use advanced AMQP features, yes why not use the
> "Almighty" API :)

Having a fully promoted, public API is a big deal. It needs full
documentation, examples and people expect a long term commitment to
it. That is ignoring any confusion you are creaeting among users by
having two APIs.

> > We need to have a discussion about our API strategy. Without getting
> > agreement on that now this discussion will keep coming up.
>
> [RA] The API strategy has nothing to do with the main goal of re-factoring.

Refactoring is a separate issue, yes. But you are talking about
creating a new API

> I think we had this discussion about API strategy several times over the
> last two years.
> And this is likely to continue for a while.

It's an important topic.

> [RA] Whether we advertise or not people will use what they want. A good
> example is what Hiram did to get AMQP support for ActiveMQ.
>  However I would personally opt to promote the low level API at the least.
> Robert Godfrey likes to see Qpid java client as well (I am neutral here).
> Let people use what they want.

We need a clear idea of why we are building software and how we expect
it to be used. If we don't even agree on how it should be used what
are our users to think?

RG

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

On 6/6/07, Robert Greig <ro...@gmail.com> wrote:
>
> On 06/06/07, Rajith Attapattu <ra...@gmail.com> wrote:
>
> >    I agree with Rob Godfrey, that if they want advanced AMQP features it
> is
> > likely that they will rewrite a reasonable chunk of code to get there
> > application to support the new logic. Compared to that, writing code to
> use
> > a AMQP API is not much. So what's all this fuss about?
>
> I don't think it follows that it is "not much". There is also the
> additional consideration that if someone is starting out you are
> creating a choice for them that they may not want to make or feel
> comfortable making. Maybe on day 1 they are happy with JMS but if by
> doing that they close the door to using "advanced" AMQP features what
> should they do? Start with the other "more powerful" API? You are
> effectively turning the JMS implementation into a second class API.
>
> Do not underestimate the cost of change.


[RA] Sorry I don't buy this argument at all. Every project needs to make
hard decisions and live with it.
You know this more than me. IMHO It's better to have choices. People always
feel they could have done it better after a while.
This dilemma is every where. Should I use EJB3 or Spring/Hibernate ? should
use JPA or not? Should I use log4j or not?
These are questions almost everybody runs into. Before evaluating what
options to use they need to evaluate their long term goals of the project.
If they think they might use advanced AMQP features, yes why not use the
"Almighty" API :)

This in anyway doesn't devalue JMS or make it second class. If they only
need JMS then they will use JMS. If they want more AMQP stuff then they will
use that.
It's all about using the right tool for the right job. If u want to cut
angles u use the miter saw and not the table saw (even though some allows u
to tilt the blade).
Does that mean that the table saw is second class to the miter saw ??

> 4. It is sad that folks have failed to see the main point of refactoring
> is
> > to modularize and achive a layered architecture that is easy to maintain
> and
> > modify.
>
> We need to have a discussion about our API strategy. Without getting
> agreement on that now this discussion will keep coming up.


[RA] The API strategy has nothing to do with the main goal of re-factoring.
There is enough consensus that we need more modularity and clear separation
of layers.
I think we had this discussion about API strategy several times over the
last two years.
And this is likely to continue for a while.

>    I didn't do the refactor to create another API. As Robert Godfrey
> points
> > out, when u rearrange the code in modular way it starts to look like an
> API
> > between each layers.
>
> I am sure there are many products that internally have layers that
> "look like APIs" but the question is whether you promote those APIs
> for people to build applications against.


[RA] Whether we advertise or not people will use what they want. A good
example is what Hiram did to get AMQP support for ActiveMQ.
 However I would personally opt to promote the low level API at the least.
Robert Godfrey likes to see Qpid java client as well (I am neutral here).
Let people use what they want.


RG
>

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 06/06/07, Rajith Attapattu <ra...@gmail.com> wrote:

>    I agree with Rob Godfrey, that if they want advanced AMQP features it is
> likely that they will rewrite a reasonable chunk of code to get there
> application to support the new logic. Compared to that, writing code to use
> a AMQP API is not much. So what's all this fuss about?

I don't think it follows that it is "not much". There is also the
additional consideration that if someone is starting out you are
creating a choice for them that they may not want to make or feel
comfortable making. Maybe on day 1 they are happy with JMS but if by
doing that they close the door to using "advanced" AMQP features what
should they do? Start with the other "more powerful" API? You are
effectively turning the JMS implementation into a second class API.

Do not underestimate the cost of change.

> 4. It is sad that folks have failed to see the main point of refactoring is
> to modularize and achive a layered architecture that is easy to maintain and
> modify.

We need to have a discussion about our API strategy. Without getting
agreement on that now this discussion will keep coming up.

>    I didn't do the refactor to create another API. As Robert Godfrey points
> out, when u rearrange the code in modular way it starts to look like an API
> between each layers.

I am sure there are many products that internally have layers that
"look like APIs" but the question is whether you promote those APIs
for people to build applications against.

RG

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

My comments in summary to all emails.

1. Agree with both Robs that we need to abstract configuration and get rid
of commons config as a standard dependency.
    But if people want they can add it under the covers. Just like slf4j
allows different logging mechanisms)
    The current config system allows system properties, file based config,
sensible defaults. The absence of each mechanism defaults to the one of the
right.

2. I have a very hard time understanding existing JMS applications wanting
to use AMQP features (not exposed through JMS) w/o much code changes.
    That looks like the justification for extending the JMS API (based on
comments so far).
    I agree with Rob Godfrey, that if they want advanced AMQP features it is
likely that they will rewrite a reasonable chunk of code to get there
application to support the new logic. Compared to that, writing code to use
a AMQP API is not much. So what's all this fuss about?

3. JMS is a priority and passing TCK is a priority. My argument is that the
Qpid java client should not make JMS the only goal. (I think saying not the
main goal was a bit of an overstatement from my part :) , so apologies )

4. It is sad that folks have failed to see the main point of refactoring is
to modularize and achive a layered architecture that is easy to maintain and
modify.
    I didn't do the refactor to create another API. As Robert Godfrey points
out, when u rearrange the code in modular way it starts to look like an API
between each layers.

Regards,

Rajith

On 6/6/07, Rupert Smith <ru...@googlemail.com> wrote:
>
> > You can argue for refactoring of the client codebase (as I
> > would) without saying that we need to write a new API.
>
> This is what I am for; refactoring as opposed to rewriting from scratch.
> Refactoring in order to introduce a hole in which to place the new layer.
> Starting from scratch seems like the easy option, because it avoids the
> graft of documenting the existing solution, and writing regression tests
> for
> bug fixes against it.
>
> I'm not coming from the point of view of someone who is defending his own
> work against being abandoned, after all the existing client code is not my
> work, I've only just started looking at it ;) I remain open to liking the
> design of the new client better, so I'm not making a judgement on one lot
> of
> code being better than the other. I'm really just thinking about what is
> best in terms of process, and capturing and carrying quality forward (in
> terms of bug fixes, performance tweaks, features users have asked for) and
> maturing the code.
>
> Ideally a release branch would undergo a contained period of bug-fixing,
> with fixes being merged onto trunk, strengthening the code on trunk. I see
> a
> conflict arising from this as much as an opportunity to resolve
> differences
> in opinion, as an irritating difference in lines of code to be resolved.
> But
> at the end of a release branch, trunk should be carrying a lot of fixes
> forward and being better for it. Otherwise, why am I bothering to merge
> changes to trunk?
>
> The M2 branch was cut prematurely, under the argument that trunk was
> needed
> for 0.9, but we knew at the time that 0.9 was going to be skipped over
> straight to 0.10. It would have been better to postpone it till things
> were
> nearing a state of completion on trunk, as this would have focussed our
> energies on completing something before forging ahead on new stuff.
>
> If the layering is so deep as to make the JMS interface notably less
> performant than the AMQ layer, then the design needs some work. I'm still
> working under the assumption that the CPU is going to spend most of its
> time
> encoding/decoding the bytes as this is not Java's forte, so the amount of
> layering may have a negligable impact, and my questions around performance
> versus layering just a red herring. The only way to know for sure is to
> take
> some measurements.
>
> We need to: make sure we don't let our customers code to non-JMS API
> features (sensible and already true in almost all cases), make sure the
> bug
> fixes that are in M2 have tests (true in many cases, I have a feeling
> we'll
> be writing some more regression tests soon).
>
> Also, I want to be following Robert Godfreys lead on the architecture,
> once
> he is free from the 0.10 spec, and once we've nailed M2.
>
> > Continuous innovation is the way forward. That's what people expect from
> > open source projects as opposed to proprietary projects.
>
> Yup, that's why people have come to expect such wonders as Maven 2 from
> the
> OS world. Now, if only they'd finished Maven 1 first, I thought it was
> ok...
>
> Rupert
>

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

On 6/6/07, Rupert Smith <ru...@googlemail.com> wrote:
>
> On 06/06/07, Rajith Attapattu <ra...@gmail.com> wrote:
>
> > [RA] We can always choose to be negative and bitch about maven at every
> > opportunity we get.
> > But don't forget most people who hate maven LOVES ant. Now is Ant an
> open
> > source project or not ?
> > And mind you they are still making improvements and there is still
> > innovation happening in ant.
> >
>
> I'm happy enough with Maven 1 and 2 and personally would choose either
> over
> Ant (I might not impose that choice on other people). I was really trying
> to
> make a point about the value of completing things, and maturing their
> quality through incremental change, versus throwing something away in
> favour
> of exciting new ideas.
>

Point taken. I agree with that sentiment.

-rajith

Re: on how to use the qpid java client

[Rupert Smith <ru...@googlemail.com>]

On 06/06/07, Rajith Attapattu <ra...@gmail.com> wrote:

> [RA] We can always choose to be negative and bitch about maven at every
> opportunity we get.
> But don't forget most people who hate maven LOVES ant. Now is Ant an open
> source project or not ?
> And mind you they are still making improvements and there is still
> innovation happening in ant.
>

I'm happy enough with Maven 1 and 2 and personally would choose either over
Ant (I might not impose that choice on other people). I was really trying to
make a point about the value of completing things, and maturing their
quality through incremental change, versus throwing something away in favour
of exciting new ideas.

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

On 6/6/07, Rupert Smith <ru...@googlemail.com> wrote:
>
> > You can argue for refactoring of the client codebase (as I
> > would) without saying that we need to write a new API.
>
> This is what I am for; refactoring as opposed to rewriting from scratch.
> Refactoring in order to introduce a hole in which to place the new layer.
> Starting from scratch seems like the easy option, because it avoids the
> graft of documenting the existing solution, and writing regression tests
> for
> bug fixes against it.


[RA] I (and Kim) tried to re-factor first w/o much success. Maybe we haven't
tried enough or took the wrong approach
Writing from scratch was the second option and not the first.
If somebody can re-factor the existing code to achieve the same level of
separation and modularity then it is great.
Apache decision making is based on merit and if you or Robert Greig can
re-factor the existing code satisfactorily then I think the community will
opt to keep the existing client. I waited for almost a year before I made
this attempt, but I am happy to concede if the above happens.

The M2 branch was cut prematurely, under the argument that trunk was needed
> for 0.9, but we knew at the time that 0.9 was going to be skipped over
> straight to 0.10. It would have been better to postpone it till things
> were
> nearing a state of completion on trunk, as this would have focussed our
> energies on completing something before forging ahead on new stuff.


[RA] These decisions were made collectively as a group. It may or may not
have been the right decision.
But the protocol was in such a state of flux, most plans got derailed.

If the layering is so deep as to make the JMS interface notably less
> performant than the AMQ layer, then the design needs some work. I'm still
> working under the assumption that the CPU is going to spend most of its
> time
> encoding/decoding the bytes as this is not Java's forte, so the amount of
> layering may have a negligable impact, and my questions around performance
> versus layering just a red herring. The only way to know for sure is to
> take
> some measurements.


[RA] The JMS layer introduces at least one context switch as apposed to the
low level AMQP API.
When u implement message listener etc, u need to build that threading model
on top of the AMQP API.
The difference in performance as u said is negligible compared to
encoding/decoding.

But for projects who wants to use AMQP under the covers will prefer one less
context switch if that is possible.
So instead of using JMS, they will use the AMQP low level client. (Apart
from the fact that the AMQP client provides full support to all AMQP
features)

We need to: make sure we don't let our customers code to non-JMS API
> features (sensible and already true in almost all cases), make sure the
> bug
> fixes that are in M2 have tests (true in many cases, I have a feeling
> we'll
> be writing some more regression tests soon).


[RA] People will choose what they want to use depending on their situations.

Also, I want to be following Robert Godfreys lead on the architecture, once
> he is free from the 0.10 spec, and once we've nailed M2.
>
> > Continuous innovation is the way forward. That's what people expect from
> > open source projects as opposed to proprietary projects.
>
> Yup, that's why people have come to expect such wonders as Maven 2 from
> the
> OS world. Now, if only they'd finished Maven 1 first, I thought it was
> ok...


[RA] We can always choose to be negative and bitch about maven at every
opportunity we get.
But don't forget most people who hate maven LOVES ant. Now is Ant an open
source project or not ?
And mind you they are still making improvements and there is still
innovation happening in ant.

Rupert
>

Re: on how to use the qpid java client

[Rupert Smith <ru...@googlemail.com>]

> You can argue for refactoring of the client codebase (as I
> would) without saying that we need to write a new API.

This is what I am for; refactoring as opposed to rewriting from scratch.
Refactoring in order to introduce a hole in which to place the new layer.
Starting from scratch seems like the easy option, because it avoids the
graft of documenting the existing solution, and writing regression tests for
bug fixes against it.

I'm not coming from the point of view of someone who is defending his own
work against being abandoned, after all the existing client code is not my
work, I've only just started looking at it ;) I remain open to liking the
design of the new client better, so I'm not making a judgement on one lot of
code being better than the other. I'm really just thinking about what is
best in terms of process, and capturing and carrying quality forward (in
terms of bug fixes, performance tweaks, features users have asked for) and
maturing the code.

Ideally a release branch would undergo a contained period of bug-fixing,
with fixes being merged onto trunk, strengthening the code on trunk. I see a
conflict arising from this as much as an opportunity to resolve differences
in opinion, as an irritating difference in lines of code to be resolved. But
at the end of a release branch, trunk should be carrying a lot of fixes
forward and being better for it. Otherwise, why am I bothering to merge
changes to trunk?

The M2 branch was cut prematurely, under the argument that trunk was needed
for 0.9, but we knew at the time that 0.9 was going to be skipped over
straight to 0.10. It would have been better to postpone it till things were
nearing a state of completion on trunk, as this would have focussed our
energies on completing something before forging ahead on new stuff.

If the layering is so deep as to make the JMS interface notably less
performant than the AMQ layer, then the design needs some work. I'm still
working under the assumption that the CPU is going to spend most of its time
encoding/decoding the bytes as this is not Java's forte, so the amount of
layering may have a negligable impact, and my questions around performance
versus layering just a red herring. The only way to know for sure is to take
some measurements.

We need to: make sure we don't let our customers code to non-JMS API
features (sensible and already true in almost all cases), make sure the bug
fixes that are in M2 have tests (true in many cases, I have a feeling we'll
be writing some more regression tests soon).

Also, I want to be following Robert Godfreys lead on the architecture, once
he is free from the 0.10 spec, and once we've nailed M2.

> Continuous innovation is the way forward. That's what people expect from
> open source projects as opposed to proprietary projects.

Yup, that's why people have come to expect such wonders as Maven 2 from the
OS world. Now, if only they'd finished Maven 1 first, I thought it was ok...

Rupert

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 07/06/07, Robert Godfrey <ro...@gmail.com> wrote:

> One thing I think needs to be made clear in the AMQP spec are which
> "classes" are meant for end-user use and which classes are meant only
> for use by the "library".

I think nearly everyone can unite in the opinion that the term "class"
as used in the AMQP specification is a poorly chosen one.

> I think the mapping of our JMS implementation to AMQP needs to be
> somewhat clearer...

I think that a mapping specification from the AMQP group is going to
be important (but probably not worth looking at until at least 0-11
due to the scope of changes).

RG

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

+1 , I agree that we can add extensions where necessary and where it makes
sense.
However like Rob Godfrey says not everything can be exposed in a meaningful
way.
Also the mapping from JMS to AMQP has to be more clearer as Rob says.

The AMQP API can be used to leverage the finer points which cannot be
meaningfully expressed by the JMS API.
Robert Greig has already provided some of those examples.

Regards,

Rajith

On 6/7/07, Robert Godfrey <ro...@gmail.com> wrote:
>
> > What is your view on how any AMQP-specific features should be exposed
> > to end users (in Java)?
>
> Adding extensions to JMS makes sense where we identify a common use
> case for such an extension.
>
> Your case of the immediate flag is once such case.
>
> The ability to create "destinations" which are not queues/topics but
> arbitrary (exchange,routing-key) pairs OR (queues) makes sense.  [The
> semantics of Destination, and in particular that the Destination you
> send to is the same as the Destination you receive from will be
> brokern, except for the two special cases of direct and topic
> exchanges].
>
> Use of other content classes would be "interesting" :-)
>
> Exposing certain methods, Basic.get for instance,  would have been
> difficult.
>
> One thing I think needs to be made clear in the AMQP spec are which
> "classes" are meant for end-user use and which classes are meant only
> for use by the "library".
>
> What I would like to see is that the use of certain JMS constructs do
> not have interesting side-effects.  For instance currently in Qpid the
> declaration of a consumer creates a queue, whereas the declaration of
> a producer does not...
>
> I think the mapping of our JMS implementation to AMQP needs to be
> somewhat clearer...
>
> -- Rob
>

Re: on how to use the qpid java client

[Robert Godfrey <ro...@gmail.com>]

> What is your view on how any AMQP-specific features should be exposed
> to end users (in Java)?

Adding extensions to JMS makes sense where we identify a common use
case for such an extension.

Your case of the immediate flag is once such case.

The ability to create "destinations" which are not queues/topics but
arbitrary (exchange,routing-key) pairs OR (queues) makes sense.  [The
semantics of Destination, and in particular that the Destination you
send to is the same as the Destination you receive from will be
brokern, except for the two special cases of direct and topic
exchanges].

Use of other content classes would be "interesting" :-)

Exposing certain methods, Basic.get for instance,  would have been difficult.

One thing I think needs to be made clear in the AMQP spec are which
"classes" are meant for end-user use and which classes are meant only
for use by the "library".

What I would like to see is that the use of certain JMS constructs do
not have interesting side-effects.  For instance currently in Qpid the
declaration of a consumer creates a queue, whereas the declaration of
a producer does not...

I think the mapping of our JMS implementation to AMQP needs to be
somewhat clearer...

-- Rob

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 07/06/07, Rajith Attapattu <ra...@gmail.com> wrote:

> How about we continue to use the so called extended JMS API.
> We can build that on top of the AMQP API. There will be an additonal layer
> that may look like an API (but we will not promote that).

This is definitely the way to go IMO.

> (We all agree that current code needs refactoring, so there is no issue
> there)

Yes.

> This way
>
> a) People who use JMS (existing and who wants to use JMS in the future) can
> leverage AMQP features if they want.
> b) People who are not worried about JMS can use the AMQP API to do what they
> want. (Ex. other apche projects ..etc)
> c) This way the AMQP API is not going to make the JMS API look less powefull
> as stated.

Yes, no arguments here.

RG

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

On 6/7/07, Robert Godfrey <ro...@gmail.com> wrote:
>
> I know this was aimed at Robert and not me... but...
>
> > How about we continue to use the so called extended JMS API.
> > We can build that on top of the AMQP API. There will be an additonal
> layer
> > that may look like an API (but we will not promote that).
> > (We all agree that current code needs refactoring, so there is no issue
> > there)
>
> +1 We refactor our current client to use an AMQP level API.  We keep
> at least the current level of expressivity for our JMS client.  We aim
> to build extensions to JMS to allow end users to reach all sensible
> AMQP functionality.
>
> I'm comfortable with an AMQP API... but I think we need to have a
> proper discussion on what a *public* API would look like... it should
> probably be a little more friendly than raw AMQP
> class/method/arguments :-)


[RA] If you look at the new refactored code you can identify two layers.
The interfaces defined in o.a.q.newclient.amqp and the interfaces defined in
o.a.q.api package.

These offer different levels of flexibility and convinience. It looks like
you are talking about something in between? (I stand to be corrected)
Where would u position your public API?

Another thing is we need to maintain these layers for our own sanity (call
them internal APIs if u will). So people who are looking for more
flexibility can tap into any of these layers. We don't have to promote these
as API's, but lets document them properly and identify them as possible
extention or leverage points.
(of course people will do so with explicit understanding that these may
change more frequently than an API).

thoughts??

Rajith

-- Rob
>
> >
> > This way
> >
> > a) People who use JMS (existing and who wants to use JMS in the future)
> can
> > leverage AMQP features if they want.
> > b) People who are not worried about JMS can use the AMQP API to do what
> they
> > want. (Ex. other apche projects ..etc)
> > c) This way the AMQP API is not going to make the JMS API look less
> powefull
> > as stated.
> >
> > Regards,
> >
> > Rajith
> >
> > On 6/7/07, Robert Greig <ro...@gmail.com> wrote:
> > >
> > > On 07/06/07, Robert Godfrey <ro...@gmail.com> wrote:
> > >
> > > > In the AMQP working group we have been talking about some sort of
> > > > standardisation of an AMQP API.  If such a standardisation takes
> place
> > > > I would expect each Qpid client library to implement to that
> standard.
> > > >  Before such a standardisation we obviously running a risk by
> defining
> > > > our own API that it is not substancially the same as that agreed by
> > > > the AMQP working group - then we would be having to support three
> > > > separate APIs :-(
> > >
> > > Yes, two APIs is more than enough work in my opinion and three would
> > > be an API too far.
> > >
> > > > My view remains that we should build an internal API based on AMQP
> in
> > > > terms of which we write our JMS implementation.  I would not be
> > > > encouraging our users to use that "AMQP" API until such time as AMQP
> > > > has sufficiently stabilized and established API guidlines.
> > >
> > > I agree with this.
> > >
> > > What is your view on how any AMQP-specific features should be exposed
> > > to end users (in Java)?
> > >
> > > RG
> > >
> >
>

Re: on how to use the qpid java client

[Alan Conway <ac...@redhat.com>]

The more I think about it, the more I am convinced that not only is a
direct AMQP-mapped API _not_ in conflict with full JMS access to AMQP,
it is the best and perhaps only practical way to achieve it!!

Cheers,
Alan.

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

On 6/8/07, Martin Ritchie <ri...@apache.org> wrote:
>
> On 07/06/07, Robert Godfrey <ro...@gmail.com> wrote:
> > I know this was aimed at Robert and not me... but...
> >
> > > How about we continue to use the so called extended JMS API.
> > > We can build that on top of the AMQP API. There will be an additonal
> layer
> > > that may look like an API (but we will not promote that).
> > > (We all agree that current code needs refactoring, so there is no
> issue
> > > there)
> >
> > +1 We refactor our current client to use an AMQP level API.  We keep
> > at least the current level of expressivity for our JMS client.  We aim
> > to build extensions to JMS to allow end users to reach all sensible
> > AMQP functionality.
> >
> > I'm comfortable with an AMQP API... but I think we need to have a
> > proper discussion on what a *public* API would look like... it should
> > probably be a little more friendly than raw AMQP
> > class/method/arguments :-)
> >
> > -- Rob
>
> +1 I agree with both RGs. Refactoring our current Java client is
> needed. Keeping the JMS extensions will allow for current JMS users to
> migrate to an AMQP broker and hopefully start to use its features.


[RA] Out of curiosity, how many of your customers use the AMQP features
provided by extended JMS?
What are the most sought after features? I know that immediate is one such
feature.
Can you please provide more details? This will help me a lot in shaping some
of my current work.

I have no objection to an AMQP API in the client code base but as others
> have said investing time just now in making it a public API when the
> spec is in such a state of change isn't warranted, but when the spec
> is more settled then we can reevaluate .


[RA]  As part of the re-factoring there is going to be a visible API. And as
the spec changes we have to do the modifications whether we expose the API
or not.
So I don't buy the argument about the time being invested.
Also anybody using the API will do so with the explicit understanding that
this will change. I think I repeated this sentence for the 100th time :)

The JMS extensions would ideally be picked up (or some set of exposed
> AMQP functionality) by the AMQP WG and provided in a AMQP->JMS mapping
> document as RGreig said.
>
> --
> Martin
>
> > >
> > > This way
> > >
> > > a) People who use JMS (existing and who wants to use JMS in the
> future) can
> > > leverage AMQP features if they want.
> > > b) People who are not worried about JMS can use the AMQP API to do
> what they
> > > want. (Ex. other apche projects ..etc)
> > > c) This way the AMQP API is not going to make the JMS API look less
> powefull
> > > as stated.
> > >
> > > Regards,
> > >
> > > Rajith
> > >
> > > On 6/7/07, Robert Greig <ro...@gmail.com> wrote:
> > > >
> > > > On 07/06/07, Robert Godfrey <ro...@gmail.com> wrote:
> > > >
> > > > > In the AMQP working group we have been talking about some sort of
> > > > > standardisation of an AMQP API.  If such a standardisation takes
> place
> > > > > I would expect each Qpid client library to implement to that
> standard.
> > > > >  Before such a standardisation we obviously running a risk by
> defining
> > > > > our own API that it is not substancially the same as that agreed
> by
> > > > > the AMQP working group - then we would be having to support three
> > > > > separate APIs :-(
> > > >
> > > > Yes, two APIs is more than enough work in my opinion and three would
> > > > be an API too far.
> > > >
> > > > > My view remains that we should build an internal API based on AMQP
> in
> > > > > terms of which we write our JMS implementation.  I would not be
> > > > > encouraging our users to use that "AMQP" API until such time as
> AMQP
> > > > > has sufficiently stabilized and established API guidlines.
> > > >
> > > > I agree with this.
> > > >
> > > > What is your view on how any AMQP-specific features should be
> exposed
> > > > to end users (in Java)?
> > > >
> > > > RG
>

Re: [c++] api

[Rajith Attapattu <ra...@gmail.com>]

+1 from me, eventhough I don't work on c++. :)
This way all clients will have an AMQP API.

Rajith

On 6/8/07, Gordon Sim <gs...@redhat.com> wrote:
>
> Alan Conway wrote:
> > I'm going to create a JIRA for this in C++. The C++ API has bothered me
> > for a while. It's non-standard, AMQP-ish but not quite but provides very
> > little value over a straight AMQP API.
>
> When I wrote the initial c++ client, I hadn't seen the python version.
> If I had, I think I would have done a few things differently.
>
> I agree it could be a cleaner and more direct expression of AMQP and I
> would fully support a move in that direction.
>

[c++] api

[Gordon Sim <gs...@redhat.com>]

Alan Conway wrote:
> I'm going to create a JIRA for this in C++. The C++ API has bothered me
> for a while. It's non-standard, AMQP-ish but not quite but provides very
> little value over a straight AMQP API.

When I wrote the initial c++ client, I hadn't seen the python version. 
If I had, I think I would have done a few things differently.

I agree it could be a cleaner and more direct expression of AMQP and I 
would fully support a move in that direction.

h

[Alan Conway <ac...@redhat.com>]

On Fri, 2007-06-08 at 11:27 -0400, Jonathan Robie wrote:
> Robert Godfrey wrote:
> > The arrival of messages from a consumer are essentially events...
> > which presumably should be handled by some sort of listener... that
> > concept doesn't occur in the API... how does the python client handle
> > it?
> 
> Python didn't used to have a listener, but it does now.
> 
> I'm attaching a working Python consumer app that shows how to use a 
> listener.
> 
> Jonathan

The python client handled consume originally by having a map of queues
named after consumers. Message for consumer "foo" goes in queue["foo"]

Adding a listener object is a recent addition and a perfect example of
how to build an AMQP API. Start with a straight mapping of AMQP
classes/methods. Add the hidden machinery to make it work, then sprinkle
on some convenience classes/functions to tie it all together. There's no
need to design a complicated "AMQP API", AMQP already does (almost)
define an API.

The main issue here is that the API changes as AMQP changes but I
wouldn't over-estimate it. The C++ API is a sort of "AMQP-ish" API that
survived the 0-9 to 0-10 shift. However if we had-had a pure AMQP API we
could equally have survived the shift by re-implementing the 0-8 classes
in terms of 0-9 operations. With the exception of the Basic Message it
would have been trivial. 

I'm going to create a JIRA for this in C++. The C++ API has bothered me
for a while. It's non-standard, AMQP-ish but not quite but provides very
little value over a straight AMQP API.

Cheers,
Alan.

Re: on how to use the qpid java client

[Jonathan Robie <jo...@redhat.com>]

Robert Godfrey wrote:
> The arrival of messages from a consumer are essentially events...
> which presumably should be handled by some sort of listener... that
> concept doesn't occur in the API... how does the python client handle
> it?

Python didn't used to have a listener, but it does now.

I'm attaching a working Python consumer app that shows how to use a 
listener.

Jonathan

Re: on how to use the qpid java client

[Robert Godfrey <ro...@gmail.com>]

On 08/06/07, Jonathan Robie <jo...@redhat.com> wrote:
> Alan Conway wrote:
> > Not so fast! The python client is living proof that raw AMQP
> > class/methods ARE quite a usable API.
>
> Absolutely!
>
> > Moreover I'd strongly argue that a
> > clean AMQP class/method layer in each project is an essential part of
> > converging our implementations and will make development of higher level
> > features dramatically easier. (C++ is not so hot in this dept. either.)
> >
>
> Having an API layer that maps directly onto the protocol seems like an
> absolute no-brainer. It makes everything easier to understand and to
> test, giving us equivalent calls that we can write tests to across
> implementations and languages.
>
> > I'm not arguing that we couldn't/shouldn't build a more friendly,
> > non-JMS Qpid API but I think todays focus should be on:
> >  - clean AMQP layer in each project
> >  - JMS (for Java) layered over this.
> >
> > Note that *both* proposed APIs for Java are based on a standard (AMQP or
> > JMS) so I'm not just making stuff up.
> >
>
> Reasoning from the examples I've been reading and writing, I suspect
> that the main difference between the "friendly" API and the more literal
> API will be in initialization and cleanup. If possible, a
> friendly-but-literal API would be really useful.

The arrival of messages from a consumer are essentially events...
which presumably should be handled by some sort of listener... that
concept doesn't occur in the API... how does the python client handle
it?

-- Rob

Re: on how to use the qpid java client

[Jonathan Robie <jo...@redhat.com>]

Alan Conway wrote:
> Not so fast! The python client is living proof that raw AMQP
> class/methods ARE quite a usable API. 

Absolutely!

> Moreover I'd strongly argue that a
> clean AMQP class/method layer in each project is an essential part of
> converging our implementations and will make development of higher level
> features dramatically easier. (C++ is not so hot in this dept. either.)
>   

Having an API layer that maps directly onto the protocol seems like an 
absolute no-brainer. It makes everything easier to understand and to 
test, giving us equivalent calls that we can write tests to across 
implementations and languages.

> I'm not arguing that we couldn't/shouldn't build a more friendly,
> non-JMS Qpid API but I think todays focus should be on:
>  - clean AMQP layer in each project
>  - JMS (for Java) layered over this.
>
> Note that *both* proposed APIs for Java are based on a standard (AMQP or
> JMS) so I'm not just making stuff up.
>   

Reasoning from the examples I've been reading and writing, I suspect 
that the main difference between the "friendly" API and the more literal 
API will be in initialization and cleanup. If possible, a 
friendly-but-literal API would be really useful.

Jonathan

Re: on how to use the qpid java client

[Alan Conway <ac...@redhat.com>]

On Fri, 2007-06-08 at 13:30 -0400, Rajith Attapattu wrote:
> >Also as Rob points out there is only a certain amount of features that u can
> expose through an extended JMS API. So I don't want to force our customers
> to use JMS when they are willing to use something else.
> For example some people like to use the message.get functionality. This
> cannot be expressed in JMS as rob points out.

I disagree. I don't know JMS at all but I have extended Java APIs before
and the magic of casting can get around most any problem:

Here's some JMS code I cogged from the web:

        // Look up a JMS topic
        Topic chatTopic = (Topic)jndi.lookup(topicName);

        // Create a JMS publisher and subscriber
        TopicPublisher publisher = 
            pubSession.createPublisher(chatTopic);
        TopicSubscriber subscriber = 
            subSession.createSubscriber(chatTopic);

        // Set a JMS message listener
        subscriber.setMessageListener(this);

Here's an imaginary way of extending it to use Qpid features like get():

 	// Drill down to the Qpid layer:
	org.apache.qpid.Subscriber qsub = ((QpidSubscriber)subscriber)

	// Access the python-style AMQP-mapped API
 	qpid.amqp0-10.Channel ch = qsub.getChannel();

	// And use it:
	qpid.amqp.0-10.Message = ch.message.get(qsub.getTicket(),
gsub.getQueue(), gsub.getDestination(), false)


No doubt I've mangled the mapping but you get the idea:
 - provide a full AMQP-mapped, python like API.
 - build JMS on top of that API 
 - use casting to get to the Qpid implementation of JMS objects.
 - query the Qpid implementation for the AMQP API objects you need.
 - use the AMQP API to do anyting AMQP can do. 
 - go back to JMS programming whenever you feel like it.

It's not completely trivial but it is relatively straightforward.
Clearly if JSM objects are implemented in terms of AMQP objects, they
have access to those objects, so we can just pass that access on to the
user want to use JMS primarily but needs to "escape" from JMS for some
special AMQP feature.

This approach to defining our APIs is almost laughably easy: both our
APIs (JMS and AMQP) are _already defined_ by the standards (and the
python client:), we just have to implement them and link them together
properly.

Cheers,
Alan.

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

>
> I *suspect* that an AMQP API will serve the needs of most anti-JMS
> customers, or at least be enough for them to get started while we figure
> out what (if any) other higher-level API features we want. If not the
> AMQP layer will make it *much* easier to implement and maintain multiple
> "personality" layers on top.
>
> Cheers,
> Alan.
>
>
> Just to add to Alans note. There are several folks who want to utilise the
full power of AMQP and are not too crazy about AMQP.
I don't like to tell our customers that they have to wait till we stabilize
the API. These early adopters are doing us a favor by showing some faith in
AMQP and Qpid.
Why do we want to discourage them ??

Also as Rob points out there is only a certain amount of features that u can
expose through an extended JMS API. So I don't want to force our customers
to use JMS when they are willing to use something else.
For example some people like to use the message.get functionality. This
cannot be expressed in JMS as rob points out.

On the same token I agree that people who want to use JMS should be able to,
and we should allow extensions where it is meaningful.

I think we have agreement here.

Regards,

Rajith

Re: on how to use the qpid java client

[Alan Conway <ac...@redhat.com>]

On Fri, 2007-06-08 at 08:25 +0100, Martin Ritchie wrote:
> On 07/06/07, Robert Godfrey <ro...@gmail.com> wrote:
> > I know this was aimed at Robert and not me... but...
> >
> > > How about we continue to use the so called extended JMS API.
> > > We can build that on top of the AMQP API. There will be an additonal layer
> > > that may look like an API (but we will not promote that).
> > > (We all agree that current code needs refactoring, so there is no issue
> > > there)
> >
> > +1 We refactor our current client to use an AMQP level API.  We keep
> > at least the current level of expressivity for our JMS client.  We aim
> > to build extensions to JMS to allow end users to reach all sensible
> > AMQP functionality.
> >
> > I'm comfortable with an AMQP API... but I think we need to have a
> > proper discussion on what a *public* API would look like... it should
> > probably be a little more friendly than raw AMQP
> > class/method/arguments :-)
> >
> > -- Rob

Not so fast! The python client is living proof that raw AMQP
class/methods ARE quite a usable API. Moreover I'd strongly argue that a
clean AMQP class/method layer in each project is an essential part of
converging our implementations and will make development of higher level
features dramatically easier. (C++ is not so hot in this dept. either.)

I'm not arguing that we couldn't/shouldn't build a more friendly,
non-JMS Qpid API but I think todays focus should be on:
 - clean AMQP layer in each project
 - JMS (for Java) layered over this.

Note that *both* proposed APIs for Java are based on a standard (AMQP or
JMS) so I'm not just making stuff up.

I *suspect* that an AMQP API will serve the needs of most anti-JMS
customers, or at least be enough for them to get started while we figure
out what (if any) other higher-level API features we want. If not the
AMQP layer will make it *much* easier to implement and maintain multiple
"personality" layers on top.

Cheers,
Alan.

Re: on how to use the qpid java client

[Martin Ritchie <ri...@apache.org>]

On 07/06/07, Robert Godfrey <ro...@gmail.com> wrote:
> I know this was aimed at Robert and not me... but...
>
> > How about we continue to use the so called extended JMS API.
> > We can build that on top of the AMQP API. There will be an additonal layer
> > that may look like an API (but we will not promote that).
> > (We all agree that current code needs refactoring, so there is no issue
> > there)
>
> +1 We refactor our current client to use an AMQP level API.  We keep
> at least the current level of expressivity for our JMS client.  We aim
> to build extensions to JMS to allow end users to reach all sensible
> AMQP functionality.
>
> I'm comfortable with an AMQP API... but I think we need to have a
> proper discussion on what a *public* API would look like... it should
> probably be a little more friendly than raw AMQP
> class/method/arguments :-)
>
> -- Rob

+1 I agree with both RGs. Refactoring our current Java client is
needed. Keeping the JMS extensions will allow for current JMS users to
migrate to an AMQP broker and hopefully start to use its features. I
have no objection to an AMQP API in the client code base but as others
have said investing time just now in making it a public API when the
spec is in such a state of change isn't warranted, but when the spec
is more settled then we can reevaluate .

The JMS extensions would ideally be picked up (or some set of exposed
AMQP functionality) by the AMQP WG and provided in a AMQP->JMS mapping
document as RGreig said.

-- 
Martin

> >
> > This way
> >
> > a) People who use JMS (existing and who wants to use JMS in the future) can
> > leverage AMQP features if they want.
> > b) People who are not worried about JMS can use the AMQP API to do what they
> > want. (Ex. other apche projects ..etc)
> > c) This way the AMQP API is not going to make the JMS API look less powefull
> > as stated.
> >
> > Regards,
> >
> > Rajith
> >
> > On 6/7/07, Robert Greig <ro...@gmail.com> wrote:
> > >
> > > On 07/06/07, Robert Godfrey <ro...@gmail.com> wrote:
> > >
> > > > In the AMQP working group we have been talking about some sort of
> > > > standardisation of an AMQP API.  If such a standardisation takes place
> > > > I would expect each Qpid client library to implement to that standard.
> > > >  Before such a standardisation we obviously running a risk by defining
> > > > our own API that it is not substancially the same as that agreed by
> > > > the AMQP working group - then we would be having to support three
> > > > separate APIs :-(
> > >
> > > Yes, two APIs is more than enough work in my opinion and three would
> > > be an API too far.
> > >
> > > > My view remains that we should build an internal API based on AMQP in
> > > > terms of which we write our JMS implementation.  I would not be
> > > > encouraging our users to use that "AMQP" API until such time as AMQP
> > > > has sufficiently stabilized and established API guidlines.
> > >
> > > I agree with this.
> > >
> > > What is your view on how any AMQP-specific features should be exposed
> > > to end users (in Java)?
> > >
> > > RG

Re: on how to use the qpid java client

[Robert Godfrey <ro...@gmail.com>]

I know this was aimed at Robert and not me... but...

> How about we continue to use the so called extended JMS API.
> We can build that on top of the AMQP API. There will be an additonal layer
> that may look like an API (but we will not promote that).
> (We all agree that current code needs refactoring, so there is no issue
> there)

+1 We refactor our current client to use an AMQP level API.  We keep
at least the current level of expressivity for our JMS client.  We aim
to build extensions to JMS to allow end users to reach all sensible
AMQP functionality.

I'm comfortable with an AMQP API... but I think we need to have a
proper discussion on what a *public* API would look like... it should
probably be a little more friendly than raw AMQP
class/method/arguments :-)

-- Rob

>
> This way
>
> a) People who use JMS (existing and who wants to use JMS in the future) can
> leverage AMQP features if they want.
> b) People who are not worried about JMS can use the AMQP API to do what they
> want. (Ex. other apche projects ..etc)
> c) This way the AMQP API is not going to make the JMS API look less powefull
> as stated.
>
> Regards,
>
> Rajith
>
> On 6/7/07, Robert Greig <ro...@gmail.com> wrote:
> >
> > On 07/06/07, Robert Godfrey <ro...@gmail.com> wrote:
> >
> > > In the AMQP working group we have been talking about some sort of
> > > standardisation of an AMQP API.  If such a standardisation takes place
> > > I would expect each Qpid client library to implement to that standard.
> > >  Before such a standardisation we obviously running a risk by defining
> > > our own API that it is not substancially the same as that agreed by
> > > the AMQP working group - then we would be having to support three
> > > separate APIs :-(
> >
> > Yes, two APIs is more than enough work in my opinion and three would
> > be an API too far.
> >
> > > My view remains that we should build an internal API based on AMQP in
> > > terms of which we write our JMS implementation.  I would not be
> > > encouraging our users to use that "AMQP" API until such time as AMQP
> > > has sufficiently stabilized and established API guidlines.
> >
> > I agree with this.
> >
> > What is your view on how any AMQP-specific features should be exposed
> > to end users (in Java)?
> >
> > RG
> >
>

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

Robert Greig,

How about we continue to use the so called extended JMS API.
We can build that on top of the AMQP API. There will be an additonal layer
that may look like an API (but we will not promote that).
(We all agree that current code needs refactoring, so there is no issue
there)

This way

a) People who use JMS (existing and who wants to use JMS in the future) can
leverage AMQP features if they want.
b) People who are not worried about JMS can use the AMQP API to do what they
want. (Ex. other apche projects ..etc)
c) This way the AMQP API is not going to make the JMS API look less powefull
as stated.

Regards,

Rajith

On 6/7/07, Robert Greig <ro...@gmail.com> wrote:
>
> On 07/06/07, Robert Godfrey <ro...@gmail.com> wrote:
>
> > In the AMQP working group we have been talking about some sort of
> > standardisation of an AMQP API.  If such a standardisation takes place
> > I would expect each Qpid client library to implement to that standard.
> >  Before such a standardisation we obviously running a risk by defining
> > our own API that it is not substancially the same as that agreed by
> > the AMQP working group - then we would be having to support three
> > separate APIs :-(
>
> Yes, two APIs is more than enough work in my opinion and three would
> be an API too far.
>
> > My view remains that we should build an internal API based on AMQP in
> > terms of which we write our JMS implementation.  I would not be
> > encouraging our users to use that "AMQP" API until such time as AMQP
> > has sufficiently stabilized and established API guidlines.
>
> I agree with this.
>
> What is your view on how any AMQP-specific features should be exposed
> to end users (in Java)?
>
> RG
>

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 07/06/07, Robert Godfrey <ro...@gmail.com> wrote:

> In the AMQP working group we have been talking about some sort of
> standardisation of an AMQP API.  If such a standardisation takes place
> I would expect each Qpid client library to implement to that standard.
>  Before such a standardisation we obviously running a risk by defining
> our own API that it is not substancially the same as that agreed by
> the AMQP working group - then we would be having to support three
> separate APIs :-(

Yes, two APIs is more than enough work in my opinion and three would
be an API too far.

> My view remains that we should build an internal API based on AMQP in
> terms of which we write our JMS implementation.  I would not be
> encouraging our users to use that "AMQP" API until such time as AMQP
> has sufficiently stabilized and established API guidlines.

I agree with this.

What is your view on how any AMQP-specific features should be exposed
to end users (in Java)?

RG

Re: AMQP Standard API?

[Gordon Sim <gs...@redhat.com>]

Jonathan Robie wrote:
> Regardless, I think the first step is to get consistency across 
> sub-projects, including an AMQP Java API, serving as an implementation 
> layer for our JMS. At that point, we might have something clean enough 
> to consider standardizing. But the API can be standardized after the 
> protocol standard, which is a higher priority.

+1!

Re: AMQP Standard API?

[Jonathan Robie <jo...@redhat.com>]

Gordon Sim wrote:
> Jonathan Robie wrote:
>> I think a genuine multi-language, platform-independent API standard 
>> would be a big win, and could really change the way people do 
>> messaging applications.
>
> Personally, I think it is a standard messaging *protocol* that will 
> change the way people do messaging.

I imagine everyone on this list agrees that the standard messaging 
protocol is the highest priority.

> The API is less important to me. Where, as in the case of JMS for 
> example, there is already an established standard I think we could 
> even confuse the aims of AMQP by pushing an alternative 'standard' at 
> this stage.
> Some consistency across the various sub-projects within Qpid however 
> is something I believe would be valuable.

Regardless, I think the first step is to get consistency across 
sub-projects, including an AMQP Java API, serving as an implementation 
layer for our JMS. At that point, we might have something clean enough 
to consider standardizing. But the API can be standardized after the 
protocol standard, which is a higher priority.

Jonathan

Re: AMQP Standard API?

[Gordon Sim <gs...@redhat.com>]

Jonathan Robie wrote:
> I think a genuine multi-language, platform-independent API standard 
> would be a big win, and could really change the way people do messaging 
> applications.

Personally, I think it is a standard messaging *protocol* that will 
change the way people do messaging.

The API is less important to me. Where, as in the case of JMS for 
example, there is already an established standard I think we could even 
confuse the aims of AMQP by pushing an alternative 'standard' at this stage.

Some consistency across the various sub-projects within Qpid however is 
something I believe would be valuable.

AMQP Standard API?

[Jonathan Robie <jo...@redhat.com>]

Robert Godfrey wrote:
> Also I specifically expressed the desire to support an AMQP
> *standards* API.  What I am very wary about is offering up a *QPID*
> AMQP API to the general public as an alternative to JMS.

I agree - an implementation is not a standard, and we need both. Qpid 
could be a reference implementation, of course.

I think a genuine multi-language, platform-independent API standard 
would be a big win, and could really change the way people do messaging 
applications.

Jonathan

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

On 6/7/07, Robert Godfrey <ro...@gmail.com> wrote:
>
> On 07/06/07, Rajith Attapattu <ra...@gmail.com> wrote:
> > On 6/7/07, Robert Godfrey <ro...@gmail.com> wrote:
> > >
> > > Just to add a few thoughts...
> > >
> > > In the AMQP working group we have been talking about some sort of
> > > standardisation of an AMQP API.  If such a standardisation takes place
> > > I would expect each Qpid client library to implement to that standard.
> > > Before such a standardisation we obviously running a risk by defining
> > > our own API that it is not substancially the same as that agreed by
> > > the AMQP working group - then we would be having to support three
> > > separate APIs :-(
> >
> >
> > [RA] Robert, I always thought the protocol classes defined in AMQP is
> the
> > API.
>
> There is API and there is API... With 0-10 (and even 0-8/0-9) certain
> classes are not meant for application use, but for use within the
> library (the Connection in 0-8 for instance).  There is more of this
> in 0-10, operating at different layers.
>
> The we have things like pre-fetch windows, handling of large messages
> (deal with them as single entities or stream the message in/out)...
> These things provoke design choices.
>
> The AMQP class and method "API" is far too raw to actually be
> presented as is to an application programmer IMHO.


[RA] Well depends on what people are looking for.

> I am not sure how we can define anything different. Please correct me if I
> > have missed something.
> > The AMQP API, I have is essentially a 1-1 map of the protocol classes
> > defined. (the same route Rafi has taken in python).
> > I didn't invent anything.
> >
>
> The python client makes some decisions about how it implements message
> listeners, threading models, etc... so will your code.  These are
> (again) design decisions.  They do not follow directly from the
> protocol spec.
>
> > My view remains that we should build an internal API based on AMQP in
> > > terms of which we write our JMS implementation.  I would not be
> > > encouraging our users to use that "AMQP" API until such time as AMQP
> > > has sufficiently stabilized and established API guidlines.
> >
> >
> > [RA] There are other apache projects and we have customers who are
> willing
> > to use the API and are aware of the risks.
> > Some of these folks are not too  crazy about JMS. If they are willing to
> > use it , then let them do it.
>
> > After all if we are only JMS, what is the differentiating factor btw us
> and
> > ActiveMQ ? (ActiveMQ will also support AMQP at one point).
>
> If ActiveMQ support JMS over AMQP, that's great.
>
> Also I specifically expressed the desire to support an AMQP
> *standards* API.  What I am very wary about is offering up a *QPID*
> AMQP API to the general public as an alternative to JMS.


[RA] I am also not keen to offer a *QPID* AMQP API.
Currently my protocol level AMQP API is what I think is a substitute for a
standard AMQP API.
When the protocol group gets to define this, I will make the nessacery
changes (hopefully not much) to support that.
People who currently want to use the low level AMQP API will do so with the
explicit understanding that it will change btw versions and obviously when
we define this *standard* API.

On a seperate note, you mentioned during a call with Gordon and me about
your desire to have a Qpid java client in addition to the JMS client.
Are you still keen on that?

I'd rather like to have the AMQP API and the JMS client layered on top. We
can have several internal APIs for layering etc, but I guess we should only
promote the two I mentioned. what do u think?

-- Rob
>
> > > On 07/06/07, Jonathan Robie <jo...@redhat.com> wrote:
> > > > Martin Ritchie wrote:
> > > > > I think that not allowing AMQP functionality via an extended JMS
> API
> > > > > is a mistake. Going down this route would, IMHO, detriment AMQP.
> > > > I also think that not allowing AMQP functionality via a pure AMQ API
> is
> > > > a mistake.
> > > >
> > > > Isn't the obvious solution to have two APIs?
> > > >
> > > > And if someone has learned the AMQP model, and wants to work with
> AMQP,
> > > > why should they have to learn JMS first?
> > > >
> > > > >> > [RA]  I'd rather like to say that JMS support is a nice value
> > > > >> addition than
> > > > >> > the main goal of Qpid java.
> > > > >>
> > > > >> I find that a staggering statement. To help me understand can you
> > > > >> please explain what you think the main goal of Qpid Java is?
> > > > > I have to agree... the Qpid Java clients first goal should be JMS
> > > > > support otherwise it is just another incompatible messaging
> product
> > > > > requiring large scale re-engineering of existing code.
> > > >
> > > > This reminds me of the old arguments in the XML community about
> whether
> > > > our standards should be designed to support documents or data. If
> XML
> > > > didn't have really good support for both, it would not have
> succeeded
> > > > the way it has.
> > > >
> > > > Of course support for JMS is crucially important. But if that's all
> we
> > > > do, I think we've missed a rare opportunity to dramatically simplify
> > > > interoperable computing across languages and platforms.
> > > >
> > > > If all we care about is JMS, let's stop spending time thinking about
> > > > anything else. But of course, the write answer is to create a really
> > > > good hub that can be used for JMS and other existing and future
> systems,
> > > > but also can be used quite nicely all by itself. In standard
> computer
> > > > science terms, let's keep these systems orthogonal, but make sure
> they
> > > > work very well together.
> > > >
> > > > Jonathan
> > > >
> > >
> >
>

Re: on how to use the qpid java client

[Robert Godfrey <ro...@gmail.com>]

On 07/06/07, Rajith Attapattu <ra...@gmail.com> wrote:
> On 6/7/07, Robert Godfrey <ro...@gmail.com> wrote:
> >
> > Just to add a few thoughts...
> >
> > In the AMQP working group we have been talking about some sort of
> > standardisation of an AMQP API.  If such a standardisation takes place
> > I would expect each Qpid client library to implement to that standard.
> > Before such a standardisation we obviously running a risk by defining
> > our own API that it is not substancially the same as that agreed by
> > the AMQP working group - then we would be having to support three
> > separate APIs :-(
>
>
> [RA] Robert, I always thought the protocol classes defined in AMQP is the
> API.

There is API and there is API... With 0-10 (and even 0-8/0-9) certain
classes are not meant for application use, but for use within the
library (the Connection in 0-8 for instance).  There is more of this
in 0-10, operating at different layers.

The we have things like pre-fetch windows, handling of large messages
(deal with them as single entities or stream the message in/out)...
These things provoke design choices.

The AMQP class and method "API" is far too raw to actually be
presented as is to an application programmer IMHO.

> I am not sure how we can define anything different. Please correct me if I
> have missed something.
> The AMQP API, I have is essentially a 1-1 map of the protocol classes
> defined. (the same route Rafi has taken in python).
> I didn't invent anything.
>

The python client makes some decisions about how it implements message
listeners, threading models, etc... so will your code.  These are
(again) design decisions.  They do not follow directly from the
protocol spec.

> My view remains that we should build an internal API based on AMQP in
> > terms of which we write our JMS implementation.  I would not be
> > encouraging our users to use that "AMQP" API until such time as AMQP
> > has sufficiently stabilized and established API guidlines.
>
>
> [RA] There are other apache projects and we have customers who are willing
> to use the API and are aware of the risks.
> Some of these folks are not too  crazy about JMS. If they are willing to
> use it , then let them do it.

> After all if we are only JMS, what is the differentiating factor btw us and
> ActiveMQ ? (ActiveMQ will also support AMQP at one point).

If ActiveMQ support JMS over AMQP, that's great.

Also I specifically expressed the desire to support an AMQP
*standards* API.  What I am very wary about is offering up a *QPID*
AMQP API to the general public as an alternative to JMS.

-- Rob

> > On 07/06/07, Jonathan Robie <jo...@redhat.com> wrote:
> > > Martin Ritchie wrote:
> > > > I think that not allowing AMQP functionality via an extended JMS API
> > > > is a mistake. Going down this route would, IMHO, detriment AMQP.
> > > I also think that not allowing AMQP functionality via a pure AMQ API is
> > > a mistake.
> > >
> > > Isn't the obvious solution to have two APIs?
> > >
> > > And if someone has learned the AMQP model, and wants to work with AMQP,
> > > why should they have to learn JMS first?
> > >
> > > >> > [RA]  I'd rather like to say that JMS support is a nice value
> > > >> addition than
> > > >> > the main goal of Qpid java.
> > > >>
> > > >> I find that a staggering statement. To help me understand can you
> > > >> please explain what you think the main goal of Qpid Java is?
> > > > I have to agree... the Qpid Java clients first goal should be JMS
> > > > support otherwise it is just another incompatible messaging product
> > > > requiring large scale re-engineering of existing code.
> > >
> > > This reminds me of the old arguments in the XML community about whether
> > > our standards should be designed to support documents or data. If XML
> > > didn't have really good support for both, it would not have succeeded
> > > the way it has.
> > >
> > > Of course support for JMS is crucially important. But if that's all we
> > > do, I think we've missed a rare opportunity to dramatically simplify
> > > interoperable computing across languages and platforms.
> > >
> > > If all we care about is JMS, let's stop spending time thinking about
> > > anything else. But of course, the write answer is to create a really
> > > good hub that can be used for JMS and other existing and future systems,
> > > but also can be used quite nicely all by itself. In standard computer
> > > science terms, let's keep these systems orthogonal, but make sure they
> > > work very well together.
> > >
> > > Jonathan
> > >
> >
>

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

On 6/7/07, Robert Godfrey <ro...@gmail.com> wrote:
>
> Just to add a few thoughts...
>
> In the AMQP working group we have been talking about some sort of
> standardisation of an AMQP API.  If such a standardisation takes place
> I would expect each Qpid client library to implement to that standard.
> Before such a standardisation we obviously running a risk by defining
> our own API that it is not substancially the same as that agreed by
> the AMQP working group - then we would be having to support three
> separate APIs :-(


[RA] Robert, I always thought the protocol classes defined in AMQP is the
API.
I am not sure how we can define anything different. Please correct me if I
have missed something.
The AMQP API, I have is essentially a 1-1 map of the protocol classes
defined. (the same route Rafi has taken in python).
I didn't invent anything.

My view remains that we should build an internal API based on AMQP in
> terms of which we write our JMS implementation.  I would not be
> encouraging our users to use that "AMQP" API until such time as AMQP
> has sufficiently stabilized and established API guidlines.


[RA] There are other apache projects and we have customers who are willing
to use the API and are aware of the risks.
Some of these folks are not too  crazy about JMS. If they are willing to
use it , then let them do it.
After all if we are only JMS, what is the differentiating factor btw us and
ActiveMQ ? (ActiveMQ will also support AMQP at one point).

-- Rob
>
> On 07/06/07, Jonathan Robie <jo...@redhat.com> wrote:
> > Martin Ritchie wrote:
> > > I think that not allowing AMQP functionality via an extended JMS API
> > > is a mistake. Going down this route would, IMHO, detriment AMQP.
> > I also think that not allowing AMQP functionality via a pure AMQ API is
> > a mistake.
> >
> > Isn't the obvious solution to have two APIs?
> >
> > And if someone has learned the AMQP model, and wants to work with AMQP,
> > why should they have to learn JMS first?
> >
> > >> > [RA]  I'd rather like to say that JMS support is a nice value
> > >> addition than
> > >> > the main goal of Qpid java.
> > >>
> > >> I find that a staggering statement. To help me understand can you
> > >> please explain what you think the main goal of Qpid Java is?
> > > I have to agree... the Qpid Java clients first goal should be JMS
> > > support otherwise it is just another incompatible messaging product
> > > requiring large scale re-engineering of existing code.
> >
> > This reminds me of the old arguments in the XML community about whether
> > our standards should be designed to support documents or data. If XML
> > didn't have really good support for both, it would not have succeeded
> > the way it has.
> >
> > Of course support for JMS is crucially important. But if that's all we
> > do, I think we've missed a rare opportunity to dramatically simplify
> > interoperable computing across languages and platforms.
> >
> > If all we care about is JMS, let's stop spending time thinking about
> > anything else. But of course, the write answer is to create a really
> > good hub that can be used for JMS and other existing and future systems,
> > but also can be used quite nicely all by itself. In standard computer
> > science terms, let's keep these systems orthogonal, but make sure they
> > work very well together.
> >
> > Jonathan
> >
>

Re: on how to use the qpid java client

[Robert Godfrey <ro...@gmail.com>]

Just to add a few thoughts...

In the AMQP working group we have been talking about some sort of
standardisation of an AMQP API.  If such a standardisation takes place
I would expect each Qpid client library to implement to that standard.
 Before such a standardisation we obviously running a risk by defining
our own API that it is not substancially the same as that agreed by
the AMQP working group - then we would be having to support three
separate APIs :-(

My view remains that we should build an internal API based on AMQP in
terms of which we write our JMS implementation.  I would not be
encouraging our users to use that "AMQP" API until such time as AMQP
has sufficiently stabilized and established API guidlines.

-- Rob

On 07/06/07, Jonathan Robie <jo...@redhat.com> wrote:
> Martin Ritchie wrote:
> > I think that not allowing AMQP functionality via an extended JMS API
> > is a mistake. Going down this route would, IMHO, detriment AMQP.
> I also think that not allowing AMQP functionality via a pure AMQ API is
> a mistake.
>
> Isn't the obvious solution to have two APIs?
>
> And if someone has learned the AMQP model, and wants to work with AMQP,
> why should they have to learn JMS first?
>
> >> > [RA]  I'd rather like to say that JMS support is a nice value
> >> addition than
> >> > the main goal of Qpid java.
> >>
> >> I find that a staggering statement. To help me understand can you
> >> please explain what you think the main goal of Qpid Java is?
> > I have to agree... the Qpid Java clients first goal should be JMS
> > support otherwise it is just another incompatible messaging product
> > requiring large scale re-engineering of existing code.
>
> This reminds me of the old arguments in the XML community about whether
> our standards should be designed to support documents or data. If XML
> didn't have really good support for both, it would not have succeeded
> the way it has.
>
> Of course support for JMS is crucially important. But if that's all we
> do, I think we've missed a rare opportunity to dramatically simplify
> interoperable computing across languages and platforms.
>
> If all we care about is JMS, let's stop spending time thinking about
> anything else. But of course, the write answer is to create a really
> good hub that can be used for JMS and other existing and future systems,
> but also can be used quite nicely all by itself. In standard computer
> science terms, let's keep these systems orthogonal, but make sure they
> work very well together.
>
> Jonathan
>

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 07/06/07, Gordon Sim <gs...@redhat.com> wrote:

> I certainly don't think JMS should be positioned as the 'poor cousin'.
> We should aim to make features that AMQP offers available in sensible
> and intuitive ways (details debated on a case-by-case basis).

This is exactly my position.

> However, if we agree that having the mapping of JMS to AMQP done via a
> lower-level 'AMQP' layer then maybe there isn't much extra effort
> involved in allowing those who want to to play with that directly? As
> pointed out by others, people should be aware that the AMQP API will
> likely change with each release until 1-0 of course.

I think there is nothing wrong with this in principle. My concerns about it are:

1) It must be made very clear that it is subject to change as the
protocol changes and that no migration path may be possible

2) There is a cost to creating an API to be used by others. It takes
careful thought and documentation (both reference documentation and
examples). We must weigh up this cost in terms of what we are *not*
doing instead (opportunity cost).

RG

Re: on how to use the qpid java client

[Gordon Sim <gs...@redhat.com>]

Robert Greig wrote:
> On 07/06/07, Jonathan Robie <jo...@redhat.com> wrote:
> 
>> Isn't the obvious solution to have two APIs?
> 
> OK, so this leads back to one of my other points: are both APIs going
> to be equivalent in terms of capabilities? Is the JMS API going to be
> some poor cousin of the other API? Or are we going to use the
> "extended JMS" approach to allow full use of AMQP capabilities via an
> API that is fundamentally JMS-like.
> 
> If both APIs are going to have equivalent power, then that certainly
> is workable. But I would still question the value of it (certainly I
> would say the opportunity cost is too high).

I certainly don't think JMS should be positioned as the 'poor cousin'. 
We should aim to make features that AMQP offers available in sensible 
and intuitive ways (details debated on a case-by-case basis).

However, if we agree that having the mapping of JMS to AMQP done via a 
lower-level 'AMQP' layer then maybe there isn't much extra effort 
involved in allowing those who want to to play with that directly? As 
pointed out by others, people should be aware that the AMQP API will 
likely change with each release until 1-0 of course.

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 07/06/07, Jonathan Robie <jo...@redhat.com> wrote:

> Isn't the obvious solution to have two APIs?

OK, so this leads back to one of my other points: are both APIs going
to be equivalent in terms of capabilities? Is the JMS API going to be
some poor cousin of the other API? Or are we going to use the
"extended JMS" approach to allow full use of AMQP capabilities via an
API that is fundamentally JMS-like.

If both APIs are going to have equivalent power, then that certainly
is workable. But I would still question the value of it (certainly I
would say the opportunity cost is too high).

> Of course support for JMS is crucially important. But if that's all we
> do, I think we've missed a rare opportunity to dramatically simplify
> interoperable computing across languages and platforms.

I don't see how the API affects interop across platforms. We can
interop today between Java JMS and C++ and Python (protocol issues
notwithstanding :-)).

> If all we care about is JMS, let's stop spending time thinking about
> anything else.

The whole point of this discussion is about how we expose AMQP
functionality in Java. It is not about bog standard JMS compliance.

RG

Re: on how to use the qpid java client

[Jonathan Robie <jo...@redhat.com>]

Martin Ritchie wrote:
> I think that not allowing AMQP functionality via an extended JMS API
> is a mistake. Going down this route would, IMHO, detriment AMQP. 
I also think that not allowing AMQP functionality via a pure AMQ API is 
a mistake.

Isn't the obvious solution to have two APIs?

And if someone has learned the AMQP model, and wants to work with AMQP, 
why should they have to learn JMS first?

>> > [RA]  I'd rather like to say that JMS support is a nice value 
>> addition than
>> > the main goal of Qpid java.
>>
>> I find that a staggering statement. To help me understand can you
>> please explain what you think the main goal of Qpid Java is?
> I have to agree... the Qpid Java clients first goal should be JMS
> support otherwise it is just another incompatible messaging product
> requiring large scale re-engineering of existing code.

This reminds me of the old arguments in the XML community about whether 
our standards should be designed to support documents or data. If XML 
didn't have really good support for both, it would not have succeeded 
the way it has.

Of course support for JMS is crucially important. But if that's all we 
do, I think we've missed a rare opportunity to dramatically simplify 
interoperable computing across languages and platforms.

If all we care about is JMS, let's stop spending time thinking about 
anything else. But of course, the write answer is to create a really 
good hub that can be used for JMS and other existing and future systems, 
but also can be used quite nicely all by itself. In standard computer 
science terms, let's keep these systems orthogonal, but make sure they 
work very well together.

Jonathan

Re: on how to use the qpid java client

[Martin Ritchie <ri...@apache.org>]

On 06/06/07, Robert Greig <ro...@gmail.com> wrote:
> On 06/06/07, Rajith Attapattu <ra...@gmail.com> wrote:
> > Hey Rob , knew this was comming :)
>
> Glad I did not let you down :-)
>
> > As for the flexible part considrer the following example. If I want to send
> > the same message to several destinations (different exchanges/routing key),
> > I can use the low level client to do  a "transfer by reference" where I send
> > the content once  followed by a several transfer methods to send the
> > different message headers. Very efficient if the message is large. ex. SOAP
> > attachment. ( message.open , several mesage.append methods to send the large
> > message , x nof message.transfers and message.close)
>
> I do not see why we would not want to expose this functionality via
> JMS. From the description it sounds to me very much like it would not
> be difficult to extend JMS to support this.
>
> Why would our business users not all want this for some specialised cases?
>
> > Here are some projects that are intested in this.
> > Apache Axis2 like a AMQP transport. They already have JMS transport and most
> > people use ActiveMQ :)
>
> It does mean they have to write the transport from scratch rather than
> just tweaking their JMS one.
>
> > A user on apache synapse said in one application he likes to do specific
> > things create/delete exchanges, queues etc only. If thats the case why would
> > he have to use JMS and only use the *non* standard" extensions we added.
> > He'd rather use the lowlevel client with minimum overhead and dependencies.
>
> The question is whether that fits into an overall strategy for all our
> users. As I see it we have different users with different
> requirements:
>
> 1) people who have an existing codebase written in terms of JMS
> (including e.g. Spring JMS higher leve stuff) and want to migrate to
> Qpid with as few changes as possible
>
> 2) people who are starting from no code but their corporate policy
> dictates that messaging should use JMS (I know at least one large
> corporation like that :-)) and the apps are simple so will just use
> standard JMS
>
> 3)  people who have a codebase written in terms of JMS but are
> interested in some of the capabilities that are being designed into
> AMQP
>
> 4) people who have no codebase and don't care about the API but want
> to exploit the features of AMQP
>
> Will the people in (4) walk away if they have to use a JMS-like API? I
> understand they will walk away if the API doesn't give them the
> flexibility or performance they require but from your post you imply
> that the mere fact it is a JMS API is a deal breaker.

I think that not allowing AMQP functionality via an extended JMS API
is a mistake. Going down this route would, IMHO, detriment AMQP. We
are trying to encourage people to use AMQP to that end we should be
looking to:

1) Make existing messaging application migration as easy as possible,
so in the java world that means JMS.

2) Make it easy to utilise the new AMQP functionality from existing
applications that are trying to utilise the new functionality of AMQP.
Again this should not require a wholesale change in messaging API.

Qpid Java is the only AMQP java client that I know of so I agree we
should have the ability to utilise the low level AMQP methods. However
if we do this without providing simultaneous JMS support then the
users might as well use the Qpid C client via JNI.

> > [RA]  I'd rather like to say that JMS support is a nice value addition than
> > the main goal of Qpid java.
>
> I find that a staggering statement. To help me understand can you
> please explain what you think the main goal of Qpid Java is?

I have to agree... the Qpid Java clients first goal should be JMS
support otherwise it is just another incompatible messaging product
requiring large scale re-engineering of existing code.

> > People understand the limitations of JMS. If u are driving a morris minor
> > and want the performance/experience of a ferrari u can try to change the
> > appearance or modify the engine. But to get the real experience of a ferrari
> > u need to get the actual one.
>
> You are phrasing this in terms of performance. Are you saying that a
> different API can provide better performance than one that implements
> JMS?
>
> > Yes cost/time will be an issue. Managers always play this card. There is
> > nothing free in this world.
>
> I think there is a serious underlying question here that we as a
> project have not yet written down: what are the goals for the project?
> What is our 6 month plan?

I agree here, we should be more open with the future plans for the project.

> You are right - nothing is free - and I think that we need to
> understand much more clearly the opportunity cost of another Java API.
>
> > [RA] People using a low level protocol API do so with the explicit
> > understanding that they may need to change code if they want to support a
> > newer version of the protocol. We don't need to kill ourselves to maintain
> > some pseudo API to insulate people from this kind of change. The low level
> > protocol is a 1-1 map of the AMQP classes and if they are
> > additions/deletions/modifications then it will be reflected in the new
> > version.
>
> But if that low level protocol API is the only way of getting access
> to certain features then we are effectively steering people to using
> that API.
>
> > All the messages they produce are of type AbstractJMSMessage which
> > implements javax.jms.Message.
>
> Is that such a big deal?
>
> > [RA] This is a good idea that I want to reuse. The ability to register
> > different factories for various mime types.
>
> It is kinda there just now but has not fully made it into the public API.
>
> > [RA] We need to build a product on a solid platform that can handle changes
> > to the protocol etc.
> > The current client did have some problems in that regard. We like to see
> > this situation addressed as soon as possible before our user base grows.
>
> I think there are a couple of discussions going on here: what should
> our API look like and what should we do with the current codebase for
> the client. You can argue for refactoring of the client codebase (as I
> would) without saying that we need to write a new API.
>
> > I understand that there are quite a few early adopters in JPMC, but I also
> > think they understand the risk of working with a protocol thats changing
> > rapidly.
>
> So we should have a page on the Qpid wiki saying "Note: please
> understand the risk of using this product due to the rapidly changing
> protocol" ?
>
> > [RA] Well indiscriminate use of system properties with no documentation is
> > not an acceptable alternative.

While I agree the lack of documentation is not acceptable (holding
hand up). I would hardly say there indiscriminate use of system
properties. The Client uses 12 System properties, half of which are
used to control dependencies such as Mina, JNDI or SASL.

Property (Default) Use
STRICT_AMQP (false) Interoperability with AMQP brokers that are
strictly AMQP compliant.

STRICT_AMQP_FATAL (false) Cause an exception if performing a JMS
function against an non capable strictly AMQP compliant broker.

IMMEDIATE_PREFETCH (false) Don't start prefetching until an explicit
receive or setMessageListener is called. Workaround for Mule.

org.apache.qpid.client.verifyQueueBindingBeforePublish (true) Required
for JMS compliance. Another example of an extension to AMQP 0_8 to
provide JMS compliance.

amq.dynamicsaslregistrar.properties ()  Used to specify a non standard
SASL properties file

amqj.protocolprovider.class () Used to specify an alternative Broker
for the InVM broker.

java.naming.provider.url () Part of JNDI

Properties that can be used to adjust the Mina connection.
amqj.heartbeat.timeoutFactor
amqj.tcpNoDelay
amqj.heartbeat.delay
amqj.sendBufferSize
amqj.receiveBufferSize

When you look at the list STRICT_AMQP, IMMEDIATE_PREFETCH and
verifyQueueBindingBeforePublish are the only ones that directly affect
the client functionality.

For 99.9% of users they need not know of the existence of these
properties. True they should be (and I shall put them up) on the wiki.
However there are real needs to have them as the current Qpid Java
client is JMS focused so the priorities are passing the TCK, working
well with existing JMS tools and interoperability with other AMQP
brokers.

> You make tradeoffs based on the resources available and the
> priorities. Nobody is arguing that system properties with no
> documentation is the way to go but given the limited set of things
> that anyone would want to change (mostly these were added for testing
> out performance tweaks) it was acceptable at the time. Opportunity
> cost again.
>
> > [RA] Well this is not the proper way to do it. We are working on adding the
> > nessacery support for JMS features like queue browsing etc to the protocol.
> > So as a temporary messure it's ok.
>
> I am glad you approve.
>
> > [RA] I think this will confuse end users.
> > We have the javax.jms.Session.createQueue(String name) and if people want to
> > use AMQP stuff there is the
> > createQueue(String queueName, boolean autoDelete ....). But to use that now
> > u need to cast the jms sesson in to AMQSession.
>
> You don't need to cast to AMQSession anywhere - that is an implementation class.
>
> > I would rather people use JMS properly and if they need AMQP features they
> > can use a proper API for that. YMMV
>
> So you are saying to people with an existing codebase that they need
> to rewrite if they want any real value from using our project.
>
> RG
>




-- 
Martin Ritchie

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 07/06/07, Jonathan Robie <jo...@redhat.com> wrote:

> These are the users who have listened to our claims that AMQ is
> platform-independent and language-independent.

> Obviously, JMS is the current mainstream solution - we need to support
> JMS, and support it well. Obviously, if we have two Java APIs, they
> should probably be two APIs to the same code base.

What does that imply about those APIs in terms of features and interoperability?

> But if our message is that AMQP is just a better way of implementing
> JMS, we lose those people.

I don't think anyone has suggested that? The crux of the matter is how
to expose AMQP-specific functionality in Java where JMS is the
predominant MoM API.

> If people only care about JMS, then we should
> drop our C++, Python, and Ruby language bindings, because they are
> irrelevant.

I don't see how this follows at all? What about people who want to use
Java and C++ and interop using AMQP?

> I'm talking to major customers who simply don't care if the
> API is JMS, but do care about the ability to use scripting languages
> together with conventional languages, and want a simple, consistent API
> across languages. Shouldn't we give them that?

I think it depends what the effort of doing that is. As I have said I
think it would be a big mistake to go down the route of having two
APIs where one is less powerful.

Do people complain about JDBC?

RG

Re: on how to use the qpid java client

[Jonathan Robie <jo...@redhat.com>]

To me, a very important set of users is:

5. people who do care about the API, and want a simple, powerful API 
with no platform-specific dependencies on .Net or Java, and want to be 
able to use essentially the same API across languages (taking into 
account differences that can't easily be ironed out across languages).

These are the users who have listened to our claims that AMQ is 
platform-independent and language-independent.

Obviously, JMS is the current mainstream solution - we need to support 
JMS, and support it well. Obviously, if we have two Java APIs, they 
should probably be two APIs to the same code base.

But if our message is that AMQP is just a better way of implementing 
JMS, we lose those people. If people only care about JMS, then we should 
drop our C++, Python, and Ruby language bindings, because they are 
irrelevant. I'm talking to major customers who simply don't care if the 
API is JMS, but do care about the ability to use scripting languages 
together with conventional languages, and want a simple, consistent API 
across languages. Shouldn't we give them that?

Jonathan

Robert Greig wrote:
> The question is whether that fits into an overall strategy for all our
> users. As I see it we have different users with different
> requirements:
>
> 1) people who have an existing codebase written in terms of JMS
> (including e.g. Spring JMS higher leve stuff) and want to migrate to
> Qpid with as few changes as possible
>
> 2) people who are starting from no code but their corporate policy
> dictates that messaging should use JMS (I know at least one large
> corporation like that :-)) and the apps are simple so will just use
> standard JMS
>
> 3)  people who have a codebase written in terms of JMS but are
> interested in some of the capabilities that are being designed into
> AMQP
>
> 4) people who have no codebase and don't care about the API but want
> to exploit the features of AMQP
>
> Will the people in (4) walk away if they have to use a JMS-like API? I
> understand they will walk away if the API doesn't give them the
> flexibility or performance they require but from your post you imply
> that the mere fact it is a JMS API is a deal breaker.

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 06/06/07, Rajith Attapattu <ra...@gmail.com> wrote:
> Hey Rob , knew this was comming :)

Glad I did not let you down :-)

> As for the flexible part considrer the following example. If I want to send
> the same message to several destinations (different exchanges/routing key),
> I can use the low level client to do  a "transfer by reference" where I send
> the content once  followed by a several transfer methods to send the
> different message headers. Very efficient if the message is large. ex. SOAP
> attachment. ( message.open , several mesage.append methods to send the large
> message , x nof message.transfers and message.close)

I do not see why we would not want to expose this functionality via
JMS. From the description it sounds to me very much like it would not
be difficult to extend JMS to support this.

Why would our business users not all want this for some specialised cases?

> Here are some projects that are intested in this.
> Apache Axis2 like a AMQP transport. They already have JMS transport and most
> people use ActiveMQ :)

It does mean they have to write the transport from scratch rather than
just tweaking their JMS one.

> A user on apache synapse said in one application he likes to do specific
> things create/delete exchanges, queues etc only. If thats the case why would
> he have to use JMS and only use the *non* standard" extensions we added.
> He'd rather use the lowlevel client with minimum overhead and dependencies.

The question is whether that fits into an overall strategy for all our
users. As I see it we have different users with different
requirements:

1) people who have an existing codebase written in terms of JMS
(including e.g. Spring JMS higher leve stuff) and want to migrate to
Qpid with as few changes as possible

2) people who are starting from no code but their corporate policy
dictates that messaging should use JMS (I know at least one large
corporation like that :-)) and the apps are simple so will just use
standard JMS

3)  people who have a codebase written in terms of JMS but are
interested in some of the capabilities that are being designed into
AMQP

4) people who have no codebase and don't care about the API but want
to exploit the features of AMQP

Will the people in (4) walk away if they have to use a JMS-like API? I
understand they will walk away if the API doesn't give them the
flexibility or performance they require but from your post you imply
that the mere fact it is a JMS API is a deal breaker.

> [RA]  I'd rather like to say that JMS support is a nice value addition than
> the main goal of Qpid java.

I find that a staggering statement. To help me understand can you
please explain what you think the main goal of Qpid Java is?

> People understand the limitations of JMS. If u are driving a morris minor
> and want the performance/experience of a ferrari u can try to change the
> appearance or modify the engine. But to get the real experience of a ferrari
> u need to get the actual one.

You are phrasing this in terms of performance. Are you saying that a
different API can provide better performance than one that implements
JMS?

> Yes cost/time will be an issue. Managers always play this card. There is
> nothing free in this world.

I think there is a serious underlying question here that we as a
project have not yet written down: what are the goals for the project?
What is our 6 month plan?

You are right - nothing is free - and I think that we need to
understand much more clearly the opportunity cost of another Java API.

> [RA] People using a low level protocol API do so with the explicit
> understanding that they may need to change code if they want to support a
> newer version of the protocol. We don't need to kill ourselves to maintain
> some pseudo API to insulate people from this kind of change. The low level
> protocol is a 1-1 map of the AMQP classes and if they are
> additions/deletions/modifications then it will be reflected in the new
> version.

But if that low level protocol API is the only way of getting access
to certain features then we are effectively steering people to using
that API.

> All the messages they produce are of type AbstractJMSMessage which
> implements javax.jms.Message.

Is that such a big deal?

> [RA] This is a good idea that I want to reuse. The ability to register
> different factories for various mime types.

It is kinda there just now but has not fully made it into the public API.

> [RA] We need to build a product on a solid platform that can handle changes
> to the protocol etc.
> The current client did have some problems in that regard. We like to see
> this situation addressed as soon as possible before our user base grows.

I think there are a couple of discussions going on here: what should
our API look like and what should we do with the current codebase for
the client. You can argue for refactoring of the client codebase (as I
would) without saying that we need to write a new API.

> I understand that there are quite a few early adopters in JPMC, but I also
> think they understand the risk of working with a protocol thats changing
> rapidly.

So we should have a page on the Qpid wiki saying "Note: please
understand the risk of using this product due to the rapidly changing
protocol" ?

> [RA] Well indiscriminate use of system properties with no documentation is
> not an acceptable alternative.

You make tradeoffs based on the resources available and the
priorities. Nobody is arguing that system properties with no
documentation is the way to go but given the limited set of things
that anyone would want to change (mostly these were added for testing
out performance tweaks) it was acceptable at the time. Opportunity
cost again.

> [RA] Well this is not the proper way to do it. We are working on adding the
> nessacery support for JMS features like queue browsing etc to the protocol.
> So as a temporary messure it's ok.

I am glad you approve.

> [RA] I think this will confuse end users.
> We have the javax.jms.Session.createQueue(String name) and if people want to
> use AMQP stuff there is the
> createQueue(String queueName, boolean autoDelete ....). But to use that now
> u need to cast the jms sesson in to AMQSession.

You don't need to cast to AMQSession anywhere - that is an implementation class.

> I would rather people use JMS properly and if they need AMQP features they
> can use a proper API for that. YMMV

So you are saying to people with an existing codebase that they need
to rewrite if they want any real value from using our project.

RG

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

Hey Rob , knew this was comming :)

On 6/5/07, Robert Greig <ro...@gmail.com> wrote:
>
> On 05/06/07, Rajith Attapattu <ra...@gmail.com> wrote:
>
> > There is also some demand for a very low level protocol level client
> that
> > works at AMQP method level from other projects who want fast AMQP
> support
> > w/o bothering to use any JMS. There is also a demand for a slightly
> higher
> > level java client w/o any JMS.
>
> Where is this demand and what is the reason for it? Is this because a
> low level API must be in some way faster, or that Real Men program at
> the lowest level and JMS is for quiche eaters?


[RA] LOL , not everybody wants another dependency (in the form of JMS) added
to their project especially if it's just infrastructure and not visible to
the end users.
What they want is AMQP support and not JMS. If they wanted JMS only they do
have other choices too. Apache itself has two projects :)

A low level client w/o an additional JMS layer and a threading model can be
faster and flexible than a JMS implementation.
As for the flexible part considrer the following example. If I want to send
the same message to several destinations (different exchanges/routing key),
I can use the low level client to do  a "transfer by reference" where I send
the content once  followed by a several transfer methods to send the
different message headers. Very efficient if the message is large. ex. SOAP
attachment. ( message.open , several mesage.append methods to send the large
message , x nof message.transfers and message.close)

Here are some projects that are intested in this.
Apache Axis2 like a AMQP transport. They already have JMS transport and most
people use ActiveMQ :)
I like to add an AMQP transport with out the additional overhead of JMS and
be able to optimise for large messages.

A user on apache synapse said in one application he likes to do specific
things create/delete exchanges, queues etc only. If thats the case why would
he have to use JMS and only use the *non* standard" extensions we added.
He'd rather use the lowlevel client with minimum overhead and dependencies.

I have said this many times now so I promise this will be the last,
> but I think having two end user APIs is a mistake. What do you say to
> users who use JMS and want to take advantage of some specific AMQP
> feature? Rewrite your application to use our "other" API?


[RA]  I'd rather like to say that JMS support is a nice value addition than
the main goal of Qpid java.
We do so bcos JMS is the standard java API. If specific AMQP features are
needed then they can try to use Qpid java client or the AMQP client to do
them while continuing to use JMS.  I don't like that option unless you are
forced to add AMQP support (not expressed through JMS like creating an
exchange) to an existing app.
People understand the limitations of JMS. If u are driving a morris minor
and want the performance/experience of a ferrari u can try to change the
appearance or modify the engine. But to get the real experience of a ferrari
u need to get the actual one.

If u want to use JMS only then fine use the API. If u want JMS primarily and
some AMQP features then try to use both clients to supplement what JMS don't
offer. If u want to leverage AMQP to it's fullest extent, please consider
using the low level client or Qpid client instead of JMS.
Yes cost/time will be an issue. Managers always play this card. There is
nothing free in this world.

> While most end users prefer JMS, most infrastructure projects wants native
> > support and likes a framework like client to work with.
>
> Why is this?


[RA] Rob see my comments above.

> Adding more methods to JMS is IMHO not the correct way to expose AMQP
> > features, neither does it do any justice for the AMQP features.
>
> Why not? Can you give a concrete example?


[RA] Rob see my comments above.

> If we are trapped in the JMS mentality for long we will not evolve. It's
> > wise to have support for JMS. However it's prudent to be a bit more
> radical
> > and have an AMQP client. It certainly has enough merit to deserve it's
> own.
>
> I don't understand this argument. You accept the need for a JMS client
> but you seem to think that we are then doing our Java users a favour
> by making that some third rate distant cousin that we send a Christmas
> card to every year but otherwise shun?
>
> I have argued strongly in the past (before Qpid was open sourced)
> against having an API that too closely mirrored the protocol, because
> of the likelihood of protocol changes and the impact that would have
> on existing users. Imagine we had a wire level API (oh we could have
> called it WireAPI to pluck a name from the ether) that people were
> coding to. How would we deal with the kind of changes planned for the
> protocol? We would be faced with either having a lot of work to
> maintain the API and map it to the new protocol or tell our users
> "sorry you're just going to have to rewrite your apps".


[RA] People using a low level protocol API do so with the explicit
understanding that they may need to change code if they want to support a
newer version of the protocol. We don't need to kill ourselves to maintain
some pseudo API to insulate people from this kind of change. The low level
protocol is a 1-1 map of the AMQP classes and if they are
additions/deletions/modifications then it will be reflected in the new
version.

If u want more insurance and be insulated from these changes, then they can
use JMS. If people adhere to the JMS spec properly there is absolutely no
need to worry (provided wire format changes don't apply as all apps talk the
same version or is backward compatible).

> [RA] No the existing message factories are too tightly coupled with JMS.
> > I am proposing three generic message factories.
> > TextMessageFactory, XMLMessageFactory, BytesMessageFactory. (We can add
> more
> > in the future .. example SOAPMessageFactory).
>
> In what way are they coupled to JMS? I had envisaged there being a
> method with signature:


All the messages they produce are of type AbstractJMSMessage which
implements javax.jms.Message.

Message createMessage(String mimeType)
>
> on Session that would allow the creation of arbitrary message types
> (that had been registered).


[RA] This is a good idea that I want to reuse. The ability to register
different factories for various mime types.

> [RA] The intent is to replace the existing client. Please  read my general
> > comment about the rational behind it.
> > Several of us tried to re-factor the existing client with limited
> success.
> > Hence we decided to rewrite the client.
>
> I think it's a good time to revisit the client as we approach the 0-10
> protocol. Having a proper logical separation of the client into an
> upper and a lower layer is important to help us keen sanity while the
> protocol evolves. There are other things we should be isolating
> ourselves from too, such as MINA so that we could plug in other
> network frameworks (e.g. Grizzly).


[RA] One of the reasons for the refactor was to isolate all these aspects.
Currently u can replace the transport phase (in the AMQP client) with a
grizzly implementation w/o any implications to any layers above it.
One reason for abandoning the old client was that the MINA specific aspects
have bled all the way to the JMS layer and a seperation was very difficult.

I do however tend to agree with Joel Spolsky who argues far more
> eloquently than I can against complete rewriting in general:
> http://www.joelonsoftware.com/articles/fog0000000069.html.
>
> > Continuous innovation is the way forward. That's what people expect from
> > open source projects as opposed to proprietary projects.
>
> A product-focussed mindset is essential if we want to grow our userbase.
>
> People who want enterprise features such as more flexible
> authorization are not going to be impressed if you say "by the way, if
> you want to use these features you're going to have to spend $100k on
> code changes to your app".


[RA] We need to build a product on a solid platform that can handle changes
to the protocol etc.
The current client did have some problems in that regard. We like to see
this situation addressed as soon as possible before our user base grows.
And that why I think we need make the changes now, not later.
We are an incubating project and people understand that there will be
changes and innovation happening. We are still in the growth phase.
However once we mature we need to think more about stability etc.

How can we stabalize as a product when the underlying AMQP protocol is still
suffering from growing pains ??
I understand that there are quite a few early adopters in JPMC, but I also
think they understand the risk of working with a protocol thats changing
rapidly.

> [RA] One of the problems of the existing client is the indiscriminate use
> of
> > the system properties with virtually no documentation. The code is
> littered
> > with the use of system properties. One improvement  I made is to unify
> the
> > configuration using commons configuration. Now all configuration has
> been
> > moved to an xml file which can be documented properly and configured
> easily.
> > However u can continue to override these properties using system
> properties
> > if u will.
>
> This is an improvement to someone and another unwanted dependency to
> someone else. There was a deliberate decision not to use commons
> configuration. A client will be used in all sorts of situations,
> embedded in app servers etc. What if someone else wants to use another
> version of commons configuration that we don't support? We really need
> our own abstraction layer so that people who want to use different
> config mechanisms can do so (a bit like slf4j does for logging).


[RA] Well indiscriminate use of system properties with no documentation is
not an acceptable alternative.
We also had properties file for some config. I think people will appreciate
a unified approach to configuration that is easy to configure and modify.

But our own abstraction is important and will work on that. After Rob
Godfreys suggestion I am planning to use slf4j for logging as well.

> w.r.t amqp strict flag, what does this do? the java implementation is
> > notorious for introducing non standard features. If u set this false
> does it
> > mean we are doing something non standard ? why do we want to do that?
>
> It was added to draw the user's attention to the fact that a
> particular feature used non-standard extensions to the AMQP protocol.
> As you may recall some JMS features could not be implemented with the
> existing AMQP protocol.


[RA] Well this is not the proper way to do it. We are working on adding the
nessacery support for JMS features like queue browsing etc to the protocol.
So as a temporary messure it's ok.

> Frankly there was no API to start with, so there want be any incompatible
> > API changes. Did we have an AMQP API visible before ?? (other than a few
> > methods added as extensions to the JMS interfaces). I don't think u can
> call
> > that an API.
>
> Why not? I think it could have been extended but that is a different
> argument.


[RA] I think this will confuse end users.
We have the javax.jms.Session.createQueue(String name) and if people want to
use AMQP stuff there is the
createQueue(String queueName, boolean autoDelete ....). But to use that now
u need to cast the jms sesson in to AMQSession.

I would rather people use JMS properly and if they need AMQP features they
can use a proper API for that. YMMV

--Rajith

RG
>

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

Hey Rupert,

On 6/7/07, Rupert Smith <ru...@googlemail.com> wrote:
>
> Rajith,
>
> To be fair, the protocol is changing so much for 0.10 that there will be
> correspondingly large changes in the code, so I hope I haven't been to
> critical of your efforts, after all, its your hard work too.


[RA] thanks. My goal is to accomadate these changes with minimum impact to
the overall architecture.
Ex. The new execution sub layer, the session layer and the channel/session
relationsjip etc.
However when we change the protocol classes like Message or Session will
result in changes to the API.
These changes are likely to happen until the protocol stabilizers a bit.

I think, we will make sure we carry forward the desired bahaviour of the
> client, in terms of tests through the JMS API. I think you haven't
> implemented the JMS part for the new client yet? So the existing JMS
> client
> on trunk should possibly still be working. Once M2 is out of the way we
> can
> evolve forward, onto the new AMQ layer, with our regression tests.


[RA] I would love if you guys also participate in evolving the  Qpid client
and the JMS layer.
I will be comming for the f2f in Aug and I hope u and martin have confirmed
too.
Lets put our heads to gether and implement/improve the JMS layer.
Until then I will slowly start building the JMS layer on top of the Qpid
client.

Please tell us about the design of the new client? Lets go back up to the
> top of this thread and engage in the original discussion about the API of
> the new client. I must say that, I am intrigued by the idea of presenting
> the protocol classes and methods in raw form as an API. It will be like
> having the protocol available to code directly against.


[RA] Thats the idea. Have u had a chance to look at the design doc on the
wiki?
Also did u get a chance to review the code?  Is there any additional
informaiton required by u?
I am more than happy to improve the documentation. I also tried my best to
put comments in the code itself.

I can see some advantages of this approach. For example, it looks like you
> have defined the protocol as a set of interfaces in the
> org.apache.qpid.nclient.amqp package. If we were, for example, to have an
> in-VM transport for clients that attach directly to a broker, it would
> seem
> to me that the broker could provide an implementation of these interfaces
> that is called into directly.


[RA] Exactly. I have spoken about this with Robert too. We can improve this
API to build the lower levels of the broker too.

Rupert
>

Re: on how to use the qpid java client

[Rupert Smith <ru...@googlemail.com>]

Rajith,

To be fair, the protocol is changing so much for 0.10 that there will be
correspondingly large changes in the code, so I hope I haven't been to
critical of your efforts, after all, its your hard work too.

I think, we will make sure we carry forward the desired bahaviour of the
client, in terms of tests through the JMS API. I think you haven't
implemented the JMS part for the new client yet? So the existing JMS client
on trunk should possibly still be working. Once M2 is out of the way we can
evolve forward, onto the new AMQ layer, with our regression tests.

Please tell us about the design of the new client? Lets go back up to the
top of this thread and engage in the original discussion about the API of
the new client. I must say that, I am intrigued by the idea of presenting
the protocol classes and methods in raw form as an API. It will be like
having the protocol available to code directly against.

I can see some advantages of this approach. For example, it looks like you
have defined the protocol as a set of interfaces in the
org.apache.qpid.nclient.amqp package. If we were, for example, to have an
in-VM transport for clients that attach directly to a broker, it would seem
to me that the broker could provide an implementation of these interfaces
that is called into directly.

Rupert

Re: on how to use the qpid java client

[Martin Ritchie <ri...@apache.org>]

On 07/06/07, Rupert Smith <ru...@googlemail.com> wrote:
> On 07/06/07, Martin Ritchie <ri...@apache.org> wrote:
> >
> > Providing two distinct APIs seems like a clean way forward with new
> > applications but it doesn't seem to offer much in the way of migration
> > for existing JMS API users.
>
>
> So, how do we let a JMS application do something that is more AMQP like? For
> example, there is no way in the current JMS implementation to do JMS over a
> headers exchange. Some suggestions:
>
> 1. Use the management console to set up the headers exchange and bind a
> queue to it. The JMS just accesses this queue already set up.
> 2. Do the same thing, but you specify this in the properties that you pass
> to the JNDI context, to get your connection factory from. Not exactly a nice
> solution.
>
> Both these approaches are pure JMS compliant. If you changed to a different
> JMS product, you'd have to figure out their way of doing it.

When you change JMS provider just now your properties file is usually
useless as each JMS provider has their own proprietary way of
specifying queues/topics and connections

> 3. Provide an extension API that gives you access to AMQP features, or just
> make the lower level AMQ API available publicly.
>
> Once an app starts doing 3, its not plain JMS anymore, so can't change to a
> different JMS provider. At least its the application writers choice to move
> outside of JMS.

But they could change do a different AMQP provider and with no or tiny
changes in code or configuration as everything is standard. This is
the end goal after all. To make AMQP language independent and vendor
independent.

> Rupert
>


-- 
Martin Ritchie

Re: on how to use the qpid java client

[Jonathan Robie <jo...@redhat.com>]

Rupert Smith wrote:
> So, how do we let a JMS application do something that is more AMQP 
> like? For
> example, there is no way in the current JMS implementation to do JMS 
> over a
> headers exchange. Some suggestions:
>
> 1. Use the management console to set up the headers exchange and bind a
> queue to it. The JMS just accesses this queue already set up.
> 2. Do the same thing, but you specify this in the properties that you 
> pass
> to the JNDI context, to get your connection factory from. Not exactly 
> a nice
> solution.
>
> Both these approaches are pure JMS compliant. If you changed to a 
> different
> JMS product, you'd have to figure out their way of doing it.


And these may well be sufficient.

> 3. Provide an extension API that gives you access to AMQP features, or 
> just
> make the lower level AMQ API available publicly.
>
> Once an app starts doing 3, its not plain JMS anymore, so can't change 
> to a
> different JMS provider. At least its the application writers choice to 
> move
> outside of JMS.

I think the only clean way to do this would be to have the functionality 
added through Java Community Process, which is not an easy thing to do.

 But should that prevent us from making additional functionality 
available in the native Java AMQ binding?

Jonathan

Re: on how to use the qpid java client

[Rupert Smith <ru...@googlemail.com>]

On 07/06/07, Martin Ritchie <ri...@apache.org> wrote:
>
> Providing two distinct APIs seems like a clean way forward with new
> applications but it doesn't seem to offer much in the way of migration
> for existing JMS API users.


So, how do we let a JMS application do something that is more AMQP like? For
example, there is no way in the current JMS implementation to do JMS over a
headers exchange. Some suggestions:

1. Use the management console to set up the headers exchange and bind a
queue to it. The JMS just accesses this queue already set up.
2. Do the same thing, but you specify this in the properties that you pass
to the JNDI context, to get your connection factory from. Not exactly a nice
solution.

Both these approaches are pure JMS compliant. If you changed to a different
JMS product, you'd have to figure out their way of doing it.

3. Provide an extension API that gives you access to AMQP features, or just
make the lower level AMQ API available publicly.

Once an app starts doing 3, its not plain JMS anymore, so can't change to a
different JMS provider. At least its the application writers choice to move
outside of JMS.

Rupert

Re: on how to use the qpid java client

[Martin Ritchie <ri...@apache.org>]

On 07/06/07, Jonathan Robie <jo...@redhat.com> wrote:
> Robert Greig wrote:
> > Let me ask another question: if you came across a database startup
> > that had some special features only available if you ditched JDBC,
> > would you want to use that product? Would you invest your own money in
> > that company?
> This is a reasonably good analogy, but it argues against the point you
> are trying to make.
>
> If what I want is database functionality, and I want to be able to
> access the database from multiple platforms, I can't use just JDBC, I
> also need something that is not dependent on the Java language or the
> Java platform, so I wind up using both JDBC and ODBC, and very likely
> ADO.Net. Of course, not all databases support all of these interfaces,
> and not all of these interfaces are available on all platforms, and
> these three APIs are gratuitously different in things that could easily
> have been done the same way.
>
> Of course, in the database world, SQL implementations also differ
> markedly, but suppose SQL worked the same way across databases and we
> were designing an API to access relational databases across platforms
> and languages. Would we want to use one model for everything but Java,
> then use JDBC for Java? I think it would make much more sense to provide
> one consistent API across languages, and provide an alternative Java
> API, which would use the same code base as the native Java API.
>
> Jonathan

Providing two distinct APIs seems like a clean way forward with new
applications but it doesn't seem to offer much in the way of migration
for existing JMS API users. They may not wish or be able to re-write
their application to make use of the new AMQP features but their
application may have need for some of them. Are we really going to say
to that group of people the only way to do it is to re-write?

As long as we provide a JMS extension that allows JMS applications to
access the AMQP layer then it doesn't really matter to me what is
going on under the covers. Existing users just need an easy upgrade
path.

-- 
Martin Ritchie

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 07/06/07, Jonathan Robie <jo...@redhat.com> wrote:

> If what I want is database functionality, and I want to be able to
> access the database from multiple platforms, I can't use just JDBC, I
> also need something that is not dependent on the Java language or the
> Java platform, so I wind up using both JDBC and ODBC, and very likely
> ADO.Net.

Yes, JDBC is obviously Java-specific.

> Of course, not all databases support all of these interfaces,
> and not all of these interfaces are available on all platforms, and
> these three APIs are gratuitously different in things that could easily
> have been done the same way.

My point is that in the Java market every database that I know of
supports JDBC. And vendors such as Sybase provide vendor-specific
functionality that is not at the SQL layer via extensions to JDBC.

I don't see the different APIs for database access as a problem
particularly. Yes it might be nice if they were closer but for various
reasons they are not. I do think that any vendor in the database space
that tried to go against ADO.NET on the .NET platform or JDBC for Java
would be extremely foolish.

> Of course, in the database world, SQL implementations also differ
> markedly, but suppose SQL worked the same way across databases and we
> were designing an API to access relational databases across platforms
> and languages. Would we want to use one model for everything but Java,
> then use JDBC for Java? I think it would make much more sense to provide
> one consistent API across languages, and provide an alternative Java
> API, which would use the same code base as the native Java API.

If we were setting out with the goal of designing an API to work
across languages for accessing relational databases are you not
stating that we are designing something to replace JDBC? I don't
understand that point in that case.

RG

Re: on how to use the qpid java client

[Jonathan Robie <jo...@redhat.com>]

Robert Greig wrote:
> Let me ask another question: if you came across a database startup
> that had some special features only available if you ditched JDBC,
> would you want to use that product? Would you invest your own money in
> that company?
This is a reasonably good analogy, but it argues against the point you 
are trying to make.

If what I want is database functionality, and I want to be able to 
access the database from multiple platforms, I can't use just JDBC, I 
also need something that is not dependent on the Java language or the 
Java platform, so I wind up using both JDBC and ODBC, and very likely 
ADO.Net. Of course, not all databases support all of these interfaces, 
and not all of these interfaces are available on all platforms, and 
these three APIs are gratuitously different in things that could easily 
have been done the same way.

Of course, in the database world, SQL implementations also differ 
markedly, but suppose SQL worked the same way across databases and we 
were designing an API to access relational databases across platforms 
and languages. Would we want to use one model for everything but Java, 
then use JDBC for Java? I think it would make much more sense to provide 
one consistent API across languages, and provide an alternative Java 
API, which would use the same code base as the native Java API.

Jonathan

Re: on how to use the qpid java client

[Gordon Sim <gs...@redhat.com>]

Robert Godfrey wrote:
> For me JMS compliance is an absolute must.

Agree wholeheartedly.

> Exposing an AMQP extension via the JMS API should be demand driven until 
> 1.0

Sounds sensible.

> Writing the JMS layer in terms of an AMQP layer makes sense in terms
> of maintainability.

Again, that sounds sensible to me.

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

> Of course, "extended JMS" is also a proprietary API (unless you push the
> extensions through Java Community Process), but perhaps we can reduce
> the percent of code that uses proprietary extensions.

When originally doing this I had hoped the following would occur:

1) our extended JMS is in some way supported by the AMQP working
group, perhaps as part of the AMQP->JMS mapping specification (which
does not of course exist yet)

2) AMQP becomes a successful protocol

3) JMS 2.0 or 3.0 includes our extensions

Obviously (3) entirely depends on AMQP making it into existing popular
messaging middleware products.

RG

Re: on how to use the qpid java client

[Jonathan Robie <jo...@redhat.com>]

Robert Greig wrote:
> On 07/06/07, Jonathan Robie <jo...@redhat.com> wrote:
>> If a user wants to use JMS, how does the presence of an alternative
>> force them to do anything different than what they would do if there is
>> only one Java API?
>
> The user may want to use JMS. But may not want to preclude using
> AMQP-specific functionality at a future date or now but only in some
> parts of the codebase. If to use AMQP-specific functionality the user
> cannot use "extended JMS" then you are forcing him to choose a
> proprietary API.

Of course, "extended JMS" is also a proprietary API (unless you push the 
extensions through Java Community Process), but perhaps we can reduce 
the percent of code that uses proprietary extensions.

In the long run, I think that standardizing our APIs is the best thing 
for users, who will want standard APIs. But I think we should first come 
up with a clean API to be standardized, one that is consistent across 
language bindings.

Jonathan

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 07/06/07, Jonathan Robie <jo...@redhat.com> wrote:
> Robert Greig wrote:
> > It's forcing users to make that decision to commit to AMQP that
> > bothers me. I think you should be able to choose to use proprietary
> > features in isolated pockets of code within the context of an
> > otherwise standards-based implementation.
>
> If a user wants to use JMS, how does the presence of an alternative
> force them to do anything different than what they would do if there is
> only one Java API?

The user may want to use JMS. But may not want to preclude using
AMQP-specific functionality at a future date or now but only in some
parts of the codebase. If to use AMQP-specific functionality the user
cannot use "extended JMS" then you are forcing him to choose a
proprietary API.

RG

Re: on how to use the qpid java client

[Jonathan Robie <jo...@redhat.com>]

Robert Greig wrote:
> On 06/06/07, Robert Godfrey <ro...@gmail.com> wrote:
>> Allowing users who are committed to AMQP (rather than to JMS) to use
>> an AMQP API does not upset me in the same way it does you.
>
> It's forcing users to make that decision to commit to AMQP that
> bothers me. I think you should be able to choose to use proprietary
> features in isolated pockets of code within the context of an
> otherwise standards-based implementation.

If a user wants to use JMS, how does the presence of an alternative 
force them to do anything different than what they would do if there is 
only one Java API?

Jonathan

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 06/06/07, Robert Godfrey <ro...@gmail.com> wrote:

> However thing such as the immediate flag are at the very least
> "unstable" right now... and we are seeing problems from our internal
> adopters where using our "extensions" make it difficult for them to
> upgrade.

Having to make small changes from version to version is not
unacceptable. And if we were willing to put the effort in, deprecated
compatibility can be achieved for certain things.

> The most problematic issue for me is the mismatch between the JMS
> notion of Destination and the AMQP notions of exchanges and consumers.
>  The current implementation jumps through hoops trying to make Topics
> and Queues in JMS map to how these are implemented in AMQP... however
> other non-standard exchanges (e.g. Headers) don't match well to the
> JMS notion (IMHO).

Yes, this is the biggest mismatch that I have seen. Promoting the
concept of exchange etc to the public API is not a bad thing though
and could be used to provide a way forward. We have bent over
backwards to make it work inside the JMS Destination model mostly
because our users want that (motivated in many cases by a desire by
then to use Spring and so on).

> JDBC depends on SQL.  There is no SQL equivalent in the world of
> messaging.  Even more, without SQL, relational databases still follow
> a reasonably well defined model (tables with columns and rows) - the
> same does not hold true for the messaging space.

I was thinking more of things like the Sybase notification feature. If
you do happen to want to use that you can do so within the context of
an application that otherwise just uses JDBC (by casting to a
SybConnection IIRC).

> Exposing an AMQP extension via the JMS API should be demand driven until 1.0

Yes, I agree. We almost certainly have other fish to fry. This is why
our extensions have not been particularly aggressive so far.

> Writing the JMS layer in terms of an AMQP layer makes sense in terms
> of maintainability.

Yes, agree here too.

> Allowing users who are committed to AMQP (rather than to JMS) to use
> an AMQP API does not upset me in the same way it does you.

It's forcing users to make that decision to commit to AMQP that
bothers me. I think you should be able to choose to use proprietary
features in isolated pockets of code within the context of an
otherwise standards-based implementation.

RG

Re: on how to use the qpid java client

[Robert Godfrey <ro...@gmail.com>]

On 06/06/07, Robert Greig <ro...@gmail.com> wrote:
> On 06/06/07, Robert Godfrey <ro...@gmail.com> wrote:
>
> > 4) if people want to use advanced AMQP features they are likely
> > writing a new application to interact with other AMQP clients; not
> > using Qpid in a situation where they could replace with another JMS
> > provider.
>
> I think it depends on what those features are. For example, the
> immediate flag in 0.8 is one small feature that some of our users have
> found to be useful.
>
> The question is: are there real examples where we cannot expose these
> advanced features using an extended JMS API?

However thing such as the immediate flag are at the very least
"unstable" right now... and we are seeing problems from our internal
adopters where using our "extensions" make it difficult for them to
upgrade.

The most problematic issue for me is the mismatch between the JMS
notion of Destination and the AMQP notions of exchanges and consumers.
 The current implementation jumps through hoops trying to make Topics
and Queues in JMS map to how these are implemented in AMQP... however
other non-standard exchanges (e.g. Headers) don't match well to the
JMS notion (IMHO).

>
> > We should have an API which mirrors the AMQP protocol.
> > We should not currently publicise or recommend its use as the protocol
> > is in too great a state of flux
>
> I can see a use for that API mostly as a basis for us building tools
> and utilities but I don't think we should leave our JMS users high and
> dry.
>

I think until the protocol reaches some sort of stability, using
"extensions" carries a risk.  Making the extensions available to
end-users should be demand driven right now (again IMHO).

> > For the very reasons that Robert states about protocol flux I would
> > not spend much time trying to expose advanced AMQP features through
> > the JMS API... where is the demand?  I would revisit this decision
> > once the protocol has reached v1.0.
>
> I do think that we need to have a better grip on prioritisation -
> users are more keen right now for us to catch up on other areas. But
> we should probably answer the question "If we did have an advanced
> AMQP feature we wanted to expose how would we do it?".
>
> Let me ask another question: if you came across a database startup
> that had some special features only available if you ditched JDBC,
> would you want to use that product? Would you invest your own money in
> that company?

JDBC depends on SQL.  There is no SQL equivalent in the world of
messaging.  Even more, without SQL, relational databases still follow
a reasonably well defined model (tables with columns and rows) - the
same does not hold true for the messaging space.

For me JMS compliance is an absolute must.
Exposing an AMQP extension via the JMS API should be demand driven until 1.0
Writing the JMS layer in terms of an AMQP layer makes sense in terms
of maintainability.
Allowing users who are committed to AMQP (rather than to JMS) to use
an AMQP API does not upset me in the same way it does you.

-- Rob

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 06/06/07, Robert Godfrey <ro...@gmail.com> wrote:

> 4) if people want to use advanced AMQP features they are likely
> writing a new application to interact with other AMQP clients; not
> using Qpid in a situation where they could replace with another JMS
> provider.

I think it depends on what those features are. For example, the
immediate flag in 0.8 is one small feature that some of our users have
found to be useful.

The question is: are there real examples where we cannot expose these
advanced features using an extended JMS API?

> We should have an API which mirrors the AMQP protocol.
> We should not currently publicise or recommend its use as the protocol
> is in too great a state of flux

I can see a use for that API mostly as a basis for us building tools
and utilities but I don't think we should leave our JMS users high and
dry.

> For the very reasons that Robert states about protocol flux I would
> not spend much time trying to expose advanced AMQP features through
> the JMS API... where is the demand?  I would revisit this decision
> once the protocol has reached v1.0.

I do think that we need to have a better grip on prioritisation -
users are more keen right now for us to catch up on other areas. But
we should probably answer the question "If we did have an advanced
AMQP feature we wanted to expose how would we do it?".

Let me ask another question: if you came across a database startup
that had some special features only available if you ditched JDBC,
would you want to use that product? Would you invest your own money in
that company?

RG

Re: on how to use the qpid java client

[Robert Godfrey <ro...@gmail.com>]

OK, time to add my two small fractional currency units...

1) at least 99.9% of our users will use the standard JMS API and this
will be sufficient for their needs
2) forcing people to use commons configuration with an xml or
.properties file is a *terrible* idea.  We should be *removing*
dependencies not adding to them.
3) the client code needs to be modularized and clearly separated into
layers for maintenance reasons.
4) if people want to use advanced AMQP features they are likely
writing a new application to interact with other AMQP clients; not
using Qpid in a situation where they could replace with another JMS
provider.

My view:

We should have an API which mirrors the AMQP protocol.
We should not currently publicise or recommend its use as the protocol
is in too great a state of flux
Our JMS implementation should be written in terms of our AMQP API;
rather than randomly peopulating it with bits of code that go directly
to the framing level.



Every dependency on an external library that we add is another
(possibly insurmountable) barrier that we add for a project to adopt
qpid.  This is *particulalry* true of dependencies on common libraries
which are not 100% compatible between all released versions (this
applies to commons configuration and log4j); since these are
*precisely* the libraries that they likely already rely on but at a
different release number.  Assuming that a client application has
access to a filesystem to read configuration data is also a huge
mistake.


Overall - as I think Thomas pointed out - the client code is a bit of
a mess.  I support the idea of a refactoring, of breaking the code
into seperate layers, of one of those layers being an AMQP API.  I
would expect each layer to be in its own package.  By its very nature
this will make the API public.

For the very reasons that Robert states about protocol flux I would
not spend much time trying to expose advanced AMQP features through
the JMS API... where is the demand?  I would revisit this decision
once the protocol has reached v1.0.

Thoughts,
Rob

Re: on how to use the qpid java client

[Robert Greig <ro...@gmail.com>]

On 05/06/07, Rajith Attapattu <ra...@gmail.com> wrote:

> There is also some demand for a very low level protocol level client that
> works at AMQP method level from other projects who want fast AMQP support
> w/o bothering to use any JMS. There is also a demand for a slightly higher
> level java client w/o any JMS.

Where is this demand and what is the reason for it? Is this because a
low level API must be in some way faster, or that Real Men program at
the lowest level and JMS is for quiche eaters?

I have said this many times now so I promise this will be the last,
but I think having two end user APIs is a mistake. What do you say to
users who use JMS and want to take advantage of some specific AMQP
feature? Rewrite your application to use our "other" API?

> While most end users prefer JMS, most infrastructure projects wants native
> support and likes a framework like client to work with.

Why is this?

> Adding more methods to JMS is IMHO not the correct way to expose AMQP
> features, neither does it do any justice for the AMQP features.

Why not? Can you give a concrete example?

> If we are trapped in the JMS mentality for long we will not evolve. It's
> wise to have support for JMS. However it's prudent to be a bit more radical
> and have an AMQP client. It certainly has enough merit to deserve it's own.

I don't understand this argument. You accept the need for a JMS client
but you seem to think that we are then doing our Java users a favour
by making that some third rate distant cousin that we send a Christmas
card to every year but otherwise shun?

I have argued strongly in the past (before Qpid was open sourced)
against having an API that too closely mirrored the protocol, because
of the likelihood of protocol changes and the impact that would have
on existing users. Imagine we had a wire level API (oh we could have
called it WireAPI to pluck a name from the ether) that people were
coding to. How would we deal with the kind of changes planned for the
protocol? We would be faced with either having a lot of work to
maintain the API and map it to the new protocol or tell our users
"sorry you're just going to have to rewrite your apps".

> [RA] No the existing message factories are too tightly coupled with JMS.
> I am proposing three generic message factories.
> TextMessageFactory, XMLMessageFactory, BytesMessageFactory. (We can add more
> in the future .. example SOAPMessageFactory).

In what way are they coupled to JMS? I had envisaged there being a
method with signature:

Message createMessage(String mimeType)

on Session that would allow the creation of arbitrary message types
(that had been registered).

> [RA] The intent is to replace the existing client. Please  read my general
> comment about the rational behind it.
> Several of us tried to re-factor the existing client with limited success.
> Hence we decided to rewrite the client.

I think it's a good time to revisit the client as we approach the 0-10
protocol. Having a proper logical separation of the client into an
upper and a lower layer is important to help us keen sanity while the
protocol evolves. There are other things we should be isolating
ourselves from too, such as MINA so that we could plug in other
network frameworks (e.g. Grizzly).

I do however tend to agree with Joel Spolsky who argues far more
eloquently than I can against complete rewriting in general:
http://www.joelonsoftware.com/articles/fog0000000069.html.

> Continuous innovation is the way forward. That's what people expect from
> open source projects as opposed to proprietary projects.

A product-focussed mindset is essential if we want to grow our userbase.

People who want enterprise features such as more flexible
authorization are not going to be impressed if you say "by the way, if
you want to use these features you're going to have to spend $100k on
code changes to your app".

> [RA] One of the problems of the existing client is the indiscriminate use of
> the system properties with virtually no documentation. The code is littered
> with the use of system properties. One improvement  I made is to unify the
> configuration using commons configuration. Now all configuration has been
> moved to an xml file which can be documented properly and configured easily.
> However u can continue to override these properties using system properties
> if u will.

This is an improvement to someone and another unwanted dependency to
someone else. There was a deliberate decision not to use commons
configuration. A client will be used in all sorts of situations,
embedded in app servers etc. What if someone else wants to use another
version of commons configuration that we don't support? We really need
our own abstraction layer so that people who want to use different
config mechanisms can do so (a bit like slf4j does for logging).

> w.r.t amqp strict flag, what does this do? the java implementation is
> notorious for introducing non standard features. If u set this false does it
> mean we are doing something non standard ? why do we want to do that?

It was added to draw the user's attention to the fact that a
particular feature used non-standard extensions to the AMQP protocol.
As you may recall some JMS features could not be implemented with the
existing AMQP protocol.

> Frankly there was no API to start with, so there want be any incompatible
> API changes. Did we have an AMQP API visible before ?? (other than a few
> methods added as extensions to the JMS interfaces). I don't think u can call
> that an API.

Why not? I think it could have been extended but that is a different argument.

RG

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

Rupert, thanks for the comments. I have marked my comments with [RA]
I also invite you to look at the code for the Qpid java client and the AMQP
low level client.
There is also a design doc that explains the AMQP low level client.

[RA] Let me start with a general comment that explains the rationale behind
the new client.
We all seems to be converging on a JMS --> Qpid Client --> AMQP Framing type
of layering for our client design.
The architecture of the AMQP protocol is evolving rapidly and our existing
client design was not able to support it with out hacks and many ugly
modifications.

As you correctly pointed out, lack of clean encapsulation of
responsibilities, classes tightly coupled and absence of a proper layering
mechanism in the old client provided major obstacles in implementing
fundamental protocol changes. Ex the execution sub layer or the new session
layer or failover.
The old client is a direct implementation of JMS interfaces. AMQP on the
other hand is evolving with some nice features that cannot be exploited
using the JMS API. Also there was no way to tap any protocol level features
for more fine grained control.

There is also some demand for a very low level protocol level client that
works at AMQP method level from other projects who want fast AMQP support
w/o bothering to use any JMS. There is also a demand for a slightly higher
level java client w/o any JMS. And then finally most end user wants JMS.
While most end users prefer JMS, most infrastructure projects wants native
support and likes a framework like client to work with.

This is NOT in any way to criticize the existing client. I rather see that
as a precursor and the new client certainly reflects the all the hard work
that has gone into the existing client.

On 6/5/07, Rupert Smith <ru...@googlemail.com> wrote:
>
> On 04/06/07, Rajith Attapattu <ra...@gmail.com> wrote:
> >
> > > Rather than having to remember the format of this URL scheme, I'd
> > > prefer (overloaded) method arguments for the constitutent parts. E.g.
> > > con.connect("localhost", 5672, "test", "guest", "guest"). Or perhaps a
> > > properties object with some defined keys (i.e. similar to how JDBC
> > > connections are often configured).
>
>
> The existing AMQConnection class provides such a method...
>
>
> > > or (probably even more controversially!):
> > >
> > > session.exchangeDeclare(false, false ...);
> > > session.queueDeclare(false, false, ...);
> > > session.queueBind(QpidConstants.DIRECT_EXCHANGE_NAME, ...);
>
>
> The existing AMQSession does it this way... Length of the class is not the
> problem, undocumented code, lack of clean encapsulation of responsibilies,
> several classes all tightly bound together is. This can all be solved with
> my refactoring axe! The session implementation is always going to be long,
> just by the number of methods defined in javax.jms.Session, that by itself
> does not make it bad.


[RA]  The length of the class affects readability. I would rather read a
class with 300-500 lines of code than something with close to 2000 lines of
code.
The problem is compounded by adding more methods to the already long
javax.jms.Session. Ex queueCreate, queueBind.
Adding more methods to JMS is IMHO not the correct way to expose AMQP
features, neither does it do any justice for the AMQP features.
If we are trapped in the JMS mentality for long we will not evolve. It's
wise to have support for JMS. However it's prudent to be a bit more radical
and have an AMQP client. It certainly has enough merit to deserve it's own.

Gordon, see my above comment. Creating messages can be hidden behind
> > factories.
>
>
> Like the existing set of message factories?


[RA] No the existing message factories are too tightly coupled with JMS.
I am proposing three generic message factories.
TextMessageFactory, XMLMessageFactory, BytesMessageFactory. (We can add more
in the future .. example SOAPMessageFactory).

All JMS messages can be easily built on top of the above message factories.
We are planning some work with native support for XML messages, hence the
XMLMessageFactory.

Stream and Map messages can be built on top of the BytesMessageFactory and
frankly I have not seen any applications using these two types of messages.
That doesn't mean we shouldn't support it.

I would like to ask my question again, about what is the reason for writing
> a new client from scratch? and what is the plan for integrating it into
> the
> existing code? Is it just an experiment with a new design that will be
> used
> to influence the existing client, or is the intention that the existing
> client is to be completely scrapped and replaced?


[RA] The intent is to replace the existing client. Please  read my general
comment about the rational behind it.
Several of us tried to re-factor the existing client with limited success.
Hence we decided to rewrite the client.

I don't want to try and defend the existing code too heavily, as already
> pointed out, it is ugly in places but it seems to me that there is
> re-invention going on when there is already an adequate solution for the
> problem. Rather than have an API that I can call in a slightly different
> way
> to what is there already, I think I'd prefer some javadoc to explain what
> all the existing parameters are for. What concerns me about throwing away
> the existing solution is the time spend re-solving the same problems
> again,
> and the danger of throwing out bug fixes that have been put in as feedback
> from customers required them.


[RA] Rupert its not  an API with a different look and feel. Architecturally
it's way different.
re-invention should be used where necessary. It doesn't mean we are throwing
away the existing hard work.
Instead we are building better solutions based on the lessons we learned in
the previous outcomes.


The existing client is maturing. Probably it was coded for the success
> pathway first, where things could go wrong the original authors just threw
> an AMQException or JMSException and forgot about it. Then some of these
> failure pathways have been examined more closely, in response to testing
> and
> bug reports and time spent thinking. There are still ways this client can
> fail in unexpected ways, but it is getting there, and I feel more than
> anything it is in need of completion, by closing off these existing
> failures, and fully testing the expected error pathways as well as the
> success ones (even better, by simply removing some of the error pathways
> that cannot occurr), and by writing the javadoc. Starting from scratch
> means
> undergoing this maturation process again.


[RA] It's a risk we have to take.  However this time we have more in depth
understanding of the AMQP protocol and the direction we are talking.
This time around we have more experience with all the problems/pit falls and
solutions that the original authors did not have.

If we were complacent with horse driven carriage and spent time just
improving it instead of thinking outside the box, where would the today's
cars , air planes be ??
Continuous innovation is the way forward. That's what people expect from
open source projects as opposed to proprietary projects.

One of the disadvantages of implementing a new client from scratch in a
> completely new package is that existing code, tests for example, will not
> be
> refactored along at the same time. If you refactor, using say intellij,
> you
> will keep this code up-to-date as you go. For example, if I safe delete a
> method, I will have to go through the tests and refactor to the new API to
> get things to compile again, then re-run the tests keeping the test suite
> up-to-date with developments.


[RA] We need to add more unit and integration tests for any code we write.
So new tests will be written for the client.

Going off on a tangent, here's a simple scheme I have used succesfully in
> the past, for regression testing old bugs as code evolves - bug report
> goes
> in Jira Qpid-203, so you write a test called testQpid203 to expose the
> bug,
> so long as the test is passing the bug remains solved.
>
> Here is a list of things that are buried somewhere in the existing code,
> that a complete re-implementation might forget about, or have to re-invent
> in a very similar fashion:
>
> Hashed password implementation for SASL.


[RA] Any improvements has to be inherited while weeding out the mistakes.
Adding this would be easy, but as u said it's also easy to forget.

The ability to turn off prefetch on new connections to work with MULE (MULEs
> fault and a pitty we have to have stuff like this in our code, but thats
> the
> real world creeping in).
> ConnectionListener to listen for failovers (unfortunately we have an
> existing customer using this non-JMS feature... their risk to bear for
> being
> non-JMS, I suppose).


[RA] The new client has better infrastructure for this kind of things. It's
written with AMQP in mind and not JMS.
It has Events and State listeners to listen to protocol level method
events/states.

Existing system properties such as amq strict flag.


[RA] One of the problems of the existing client is the indiscriminate use of
the system properties with virtually no documentation. The code is littered
with the use of system properties. One improvement  I made is to unify the
configuration using commons configuration. Now all configuration has been
moved to an xml file which can be documented properly and configured easily.
However u can continue to override these properties using system properties
if u will.

w.r.t amqp strict flag, what does this do? the java implementation is
notorious for introducing non standard features. If u set this false does it
mean we are doing something non standard ? why do we want to do that?

Synchronous request/response that can be interrupted by fail-over. The way
> fail-over works will change for the new protocol, because it supports it
> better, but a similar solution involving interruptable tasks and some sort
> of retry will inevitably be invented.


[RA] Failover will anyhow have to change and I don't see how u can implement
with the existing client with out major modifications.

This is just stuff that I know of, I haven't fully examined the existing
> code, so there are probably other things too.
>
> The danger is leaving existing customers behind on an old branch, and not
> being able to bring them easily onto the new branch, for fear of
> re-introducing old bugs,



[RA] Well when we change AMQP versions this is inevitable. Introducing a new
client doesn't mean we are re-introducing old bugs.
Some of these bugs  will not even  be relevant  with the new client
architecture.

or incompatible API changes.


[RA] If customers are using JMS what is the issue?
I think u must be referring to the non JMS AMQP specific methods. That's
precisely what we are trying to address in the new client.
Frankly there was no API to start with, so there want be any incompatible
API changes. Did we have an AMQP API visible before ?? (other than a few
methods added as extensions to the JMS interfaces). I don't think u can call
that an API.


I did post about APIs
> months ago, to make the point that an API needs to be decided early and
> preferably set in stone from then on (things are a bit different for the
> .Net because there are less people using it in the wild yet).
> Unfortunately,
> you need to give some carefull consideration before changing a public API
> method, and the nice way to do it is to mark the old one @deprecated for a
> while, before removing it.
>
> Rupert
>

Re: on how to use the qpid java client

[Rupert Smith <ru...@googlemail.com>]

On 04/06/07, Rajith Attapattu <ra...@gmail.com> wrote:
>
> > Rather than having to remember the format of this URL scheme, I'd
> > prefer (overloaded) method arguments for the constitutent parts. E.g.
> > con.connect("localhost", 5672, "test", "guest", "guest"). Or perhaps a
> > properties object with some defined keys (i.e. similar to how JDBC
> > connections are often configured).


The existing AMQConnection class provides such a method...


> > or (probably even more controversially!):
> >
> > session.exchangeDeclare(false, false ...);
> > session.queueDeclare(false, false, ...);
> > session.queueBind(QpidConstants.DIRECT_EXCHANGE_NAME, ...);


The existing AMQSession does it this way... Length of the class is not the
problem, undocumented code, lack of clean encapsulation of responsibilies,
several classes all tightly bound together is. This can all be solved with
my refactoring axe! The session implementation is always going to be long,
just by the number of methods defined in javax.jms.Session, that by itself
does not make it bad.

Gordon, see my above comment. Creating messages can be hidden behind
> factories.


Like the existing set of message factories?

I would like to ask my question again, about what is the reason for writing
a new client from scratch? and what is the plan for integrating it into the
existing code? Is it just an experiment with a new design that will be used
to influence the existing client, or is the intention that the existing
client is to be completely scrapped and replaced?

I don't want to try and defend the existing code too heavily, as already
pointed out, it is ugly in places but it seems to me that there is
re-invention going on when there is already an adequate solution for the
problem. Rather than have an API that I can call in a slightly different way
to what is there already, I think I'd prefer some javadoc to explain what
all the existing parameters are for. What concerns me about throwing away
the existing solution is the time spend re-solving the same problems again,
and the danger of throwing out bug fixes that have been put in as feedback
from customers required them.

The existing client is maturing. Probably it was coded for the success
pathway first, where things could go wrong the original authors just threw
an AMQException or JMSException and forgot about it. Then some of these
failure pathways have been examined more closely, in response to testing and
bug reports and time spent thinking. There are still ways this client can
fail in unexpected ways, but it is getting there, and I feel more than
anything it is in need of completion, by closing off these existing
failures, and fully testing the expected error pathways as well as the
success ones (even better, by simply removing some of the error pathways
that cannot occurr), and by writing the javadoc. Starting from scratch means
undergoing this maturation process again.

One of the disadvantages of implementing a new client from scratch in a
completely new package is that existing code, tests for example, will not be
refactored along at the same time. If you refactor, using say intellij, you
will keep this code up-to-date as you go. For example, if I safe delete a
method, I will have to go through the tests and refactor to the new API to
get things to compile again, then re-run the tests keeping the test suite
up-to-date with developments.

Going off on a tangent, here's a simple scheme I have used succesfully in
the past, for regression testing old bugs as code evolves - bug report goes
in Jira Qpid-203, so you write a test called testQpid203 to expose the bug,
so long as the test is passing the bug remains solved.

Here is a list of things that are buried somewhere in the existing code,
that a complete re-implementation might forget about, or have to re-invent
in a very similar fashion:

Hashed password implementation for SASL.
The ability to turn off prefetch on new connections to work with MULE (MULEs
fault and a pitty we have to have stuff like this in our code, but thats the
real world creeping in).
ConnectionListener to listen for failovers (unfortunately we have an
existing customer using this non-JMS feature... their risk to bear for being
non-JMS, I suppose).
Existing system properties such as amq strict flag.
Synchronous request/response that can be interrupted by fail-over. The way
fail-over works will change for the new protocol, because it supports it
better, but a similar solution involving interruptable tasks and some sort
of retry will inevitably be invented.

This is just stuff that I know of, I haven't fully examined the existing
code, so there are probably other things too.

The danger is leaving existing customers behind on an old branch, and not
being able to bring them easily onto the new branch, for fear of
re-introducing old bugs, or incompatible API changes. I did post about APIs
months ago, to make the point that an API needs to be decided early and
preferably set in stone from then on (things are a bit different for the
.Net because there are less people using it in the wild yet). Unfortunately,
you need to give some carefull consideration before changing a public API
method, and the nice way to do it is to mark the old one @deprecated for a
while, before removing it.

Rupert

Re: on how to use the qpid java client

[Rajith Attapattu <ra...@gmail.com>]

Gordon,

Thank you very much for the feedback !!!!!
The API certainly needs some tweaking and I wasn't able to put out a design
document as promised.
I still have some unfinished work, especially the message factories. Thats
why I didn't make an official announcement.
I checked in bcos, the toronto office had a hardware failure and we don't
have access to our home directories for about a week now.
My comments are inline and thanks again for the feedback.

Regards,

Rajith

On 6/4/07, Gordon Sim <gs...@redhat.com> wrote:
>
> Gordon Sim wrote:
> > rajith@apache.org wrote:
> >> Author: rajith
> >> Date: Fri Jun  1 13:40:34 2007
> >> New Revision: 543602
> >>
> >> URL: http://svn.apache.org/viewvc?view=rev&rev=543602
> >> Log:
> >> more enchancements for the Qpid java client. Also I have checked in a
> >> sample client(QpidTestClient) on how to use the qpid java client.
> >
> > Rajith,
> >
> > Great to see this coming together! The sample was perfect for getting a
> > quick overview of how the API is to work. It will also be very useful in
> > focusing the discussion on more uniformity across the qpid clients.
> >
> > I've attached some comments on that sample. I have to confess though
> > that I didn't investigate the APIs directly so some of these comments
> > may be based on ignorance of what is already there. A lot of them are
> > also probably based on prejudices acquired in connection with the c++ or
> > python clients!
> >
> > --Gordon.
> >
>
> Many apologies; comments are now attached as originally promised!
>
>
>
>
>     QpidConnection con = new QpidConnectionImpl();
>     con.connect("amqp://guest:guest@test
> /test?brokerlist='tcp://localhost:5672?'");
>
> Rather than having to remember the format of this URL scheme, I'd
> prefer (overloaded) method arguments for the constitutent parts. E.g.
> con.connect("localhost", 5672, "test", "guest", "guest"). Or perhaps a
> properties object with some defined keys (i.e. similar to how JDBC
> connections are often configured).


+1,  I will add that additional method.  JMS was the focus and they usually
use a url.
But u have a very valid point and I am happy to add that.

    QpidSession session = con.createSession(
> QpidConstants.SESSION_EXPIRY_TIED_TO_CHANNEL);
>     session.open();
>
> Though a minor point, there should be a createSession() with no
> argument that assumes the expiry is tied to the channel. That keeps
> the simple case simple.


+1 again. Will add.

    QpidExchangeHelper exchangeHelper = session.getExchangeHelper();
>     exchangeHelper.declareExchange(false, false,
> QpidConstants.DIRECT_EXCHANGE_NAME, false, false, false,
> QpidConstants.DIRECT_EXCHANGE_CLASS);
>
>     QpidQueueHelper queueHelper = session.getQueueHelper();
>     queueHelper.declareQueue(false, false, false, false, false,
> "myQueue");
>     queueHelper.bindQueue(QpidConstants.DIRECT_EXCHANGE_NAME, false,
> "myQueue", "RH");
>
> Perhaps a bit controversial, but how about e.g.
>
> session.exchange.declare(false, false ...);
> session.queue.declare(false, false, ...);
> session.queue.bind(QpidConstants.DIRECT_EXCHANGE_NAME, ...);
>
> or (probably even more controversially!):
>
> session.exchangeDeclare(false, false ...);
> session.queueDeclare(false, false, ...);
> session.queueBind(QpidConstants.DIRECT_EXCHANGE_NAME, ...);
>
> I thought hard about option 1 & 3 (didn't think about option 2).
I stayed away from option 3 bcos,
a) The session class becomes more complex and lengthy with all the methods
added (a problem with the current code)
b) If we hide Queue, Exchange and Message stuff behind interfaces then we
can clearly isolate these areas and changes. modifications are easy managed.

I thought if I compartmentalize these functional areas, it will make the
code more simple, manageable and readable.
However I agree that option 3 looks more natural and elegant, but I choose
to compromise them for the sake of above properties, keeping in mind that it
will help with more mundane tasks of maintenance and modifications.

Option 2 looks like a nice compromise, however it is a bit controversial. If
u guys like that approach, I can promote that.
thoughts ???


Ideally the method fields would be in the order defined in the spec or
> the order of relevance; in queue.declare for example, having the name
> first would be more 'appealing' to me. We should also expose the
> 'arguments' field table.


Gordon,  the argument order is in the exact order from the generated classes
from the spec.
I will look closely again to see if I have made any mistakes. But I agree
that the name as the first argument is more natural.

Very minor point: should it not be DIRECT_EXCHANGE_TYPE, rather than
> _CLASS?


Agreed. This was a direct  (blind) copy from the existing code :)
Will change this.

    MessageHeaders msgHeaders = new MessageHeaders();
>     msgHeaders.setRoutingKey(new AMQShortString("RH"));
>     msgHeaders.setExchange(new AMQShortString(
> QpidConstants.DIRECT_EXCHANGE_NAME));
>     AMQPApplicationMessage msg = new
> AMQPApplicationMessage(msgHeaders,"test".getBytes());
>
> The application should not have to care about AMQShortString; it
> should be able to pass in a normal java string and have the client
> wrap it if required.


This is purely laziness on my part.  I just wanted to test end to end so
this is a big hack :)
If u look at the other code (Connection, session, queue, exchange) I have
wrapped all the AMQ specifics like AMQString etc with java.lang.string to
make it more natural. Only thing that I am stuck with is the field table.

I want to introduce a message factory that will take in a few parameters and
produce a message.
//ex: MessageFactory.createTextMessage(msgText,exchangeName,routingKey);
//ex: MessageFactory.createXMLMessage(xmlText,exchangeName,routingKey);
//ex: MessageFactory.createBytesMessage(bytes,exchangeName,routingKey);

The discussions with Tomas and Jonathon also indicated that they would like
message factories.

My view on the exchange is that the application wouldn't directly set
> a message header, but would specify the exchange to which a publish
> was directed. I think I would do the same for routing key also.


Gordon, see my above comment. Creating messages can be hidden behind
factories.

    QpidMessageProducer messageProducer = session.createProducer();
>     messageProducer.open();
>     messageProducer.send(false, true, msg);
>
> I would directly expose e.g. the message.transfer method, rather than
> wrapping it behind a more generic producer interface.


Two things were on my mind.  First was the JMS  API and the second and more
important point was the current thinking reflected by
Rafi and Robert Godfrey about breaking the transfer into a send and receive
(may not be the exact terminology).

    QpidMessageConsumer messageConsumer = session.createConsumer("myQueue",
> false, false);
>     messageConsumer.open();
>
>     AMQPApplicationMessage msg2 = messageConsumer.receive();
>
> Again, I think I would lean towards more direct expression of the AMQP
> methods. So a message.consume method, and then the ability to set a
> listener or poll from a queue for that consumer. e.g.
>
> QpidMessageConsumer consumer = session.message.consume("myQueue", false,
> false);
> //either
> consumer.setListener(...);
> //or
> AMQPApplicationMessage msg = consumer.receive()


Gordon, I wanted the API methods to communicate  to the end user, as much
information as possible.
session.message.consume method would give the impression that u are
referencing an already created object.
While session.createConsumer will clearly communicate that you are creating
a new consumer.

I can do consumer.open inside the create method, but an explicit
consumer.open will remind them to close the resource too.
Or else it will only be closed when the session is closed.

There is always a fine line in deciding an API. Lots of things to balance
:). But your feedback is very much appreciated.

Re: on how to use the qpid java client

[Gordon Sim <gs...@redhat.com>]

Gordon Sim wrote:
> rajith@apache.org wrote:
>> Author: rajith
>> Date: Fri Jun  1 13:40:34 2007
>> New Revision: 543602
>>
>> URL: http://svn.apache.org/viewvc?view=rev&rev=543602
>> Log:
>> more enchancements for the Qpid java client. Also I have checked in a 
>> sample client(QpidTestClient) on how to use the qpid java client.
> 
> Rajith,
> 
> Great to see this coming together! The sample was perfect for getting a 
> quick overview of how the API is to work. It will also be very useful in 
> focusing the discussion on more uniformity across the qpid clients.
> 
> I've attached some comments on that sample. I have to confess though 
> that I didn't investigate the APIs directly so some of these comments 
> may be based on ignorance of what is already there. A lot of them are 
> also probably based on prejudices acquired in connection with the c++ or 
> python clients!
> 
> --Gordon.
> 

Many apologies; comments are now attached as originally promised!