You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Kevin Smith <ks...@redhat.com> on 2007/01/26 15:33:00 UTC
YAML Configuration (Was Re: QPID-6)
Alan Conway wrote:
> On Thu, 2007-01-25 at 11:20 +0000, Andrew Stitcher wrote:
>> My preference for config file format would currently be YAML
>> (http://www.yaml.org/). As there are implementations for lots of
>> languages and the file format is easy to read and write by a person
>> (unlike XML).
>>
>> It would obviously be beneficial for the C++ broker to use the same
>> config file as the Java broker for ease of transistion so this would be
>> another consideration. However then we would presumably need to add XML
>> persing into the C++ broker (in my experience big and unwieldy) just for
>> the config file.
>
>
> yaml looks good to me, and it still lets us support the XML fetishists
> out there. From http://www.yaml.org/xml.html :
>
> "For those who love YAML, but require buzz-word compliance or require
> XML angle brackets, there is a clean option -- a subset of XML which has
> YAML's information model, but XML's syntax. Since a YAML stream may not
> start with a '<' character, a YAML implementation could implement an
> implicit, automatic conversion of XML in this schema to the equivalent
> YAML. As a proof of concept, a XSLT Stylesheet is provided, along with
> the canonical invoice example in XML using this schema.
>
> Cheers,
> Alan.
>
>
I'm planning on writing this using pure YAML for commons-configuration soonish
(over the weekend). My plan is to get this into commons-configuration proper
since they already have a bug open on this
(http://issues.apache.org/jira/browse/CONFIGURATION-201).
Thoughts?
--Kevin
Re: YAML Configuration (Was Re: QPID-6)
Posted by Alan Conway <ac...@redhat.com>.
On Fri, 2007-01-26 at 09:33 -0500, Kevin Smith wrote:
> I'm planning on writing this using pure YAML for commons-configuration soonish
> (over the weekend). My plan is to get this into commons-configuration proper
> since they already have a bug open on this
> (http://issues.apache.org/jira/browse/CONFIGURATION-201).
>
>
> Thoughts?
>
> --Kevin
Sounds great. We'll need it in C++ too but it'll be easier if there's
already something to base it on in Java.
We'll need to think about naming conventions etc. so we can share useful
info between C++ and Java without creating conflicts over
language-specific items, but that's another conversation.
Cheers,
Alan.
Re: YAML Configuration (Was Re: QPID-6)
Posted by Robert Greig <ro...@gmail.com>.
On 26/01/07, Robert Godfrey <ro...@gmail.com> wrote:
> > >
> > This is an interesting idea. We could do that for both the broker and the
> > client, assuming that there's a commons-configuration class which supports
> > the
> > user's desired format.
Maybe jarjar could be used to repackage commons-configuration so that
we can use it but not extend the client's dependency set.
commons-configuration does seem pretty good.
See http://tonicsystems.com/products/jarjar/
Has anyone used it?
RG
Re: YAML Configuration (Was Re: QPID-6)
Posted by Robert Godfrey <ro...@gmail.com>.
> >
> > Given that we cannot control the context in which the client library is
> > deployed, I think we need to be flexible in the way in which these
> > configuration options can be set. That is I think that it should be
> > possible to configure the library by passing in a
> > commons.Configurationobject. In that sense for the (Java) client I
> > think the decision over XML
> > vs YAML vs Properties file etc may be something of a red-herring.
> > However I
> > am +1 for using XML as the default and as the basis for a common
> > configuration style between clients / brokers of all languages.
>
> This is an interesting idea. We could do that for both the broker and the
> client, assuming that there's a commons-configuration class which supports
> the
> user's desired format.
I've no idea what exists in the C++ world (if anything) that gives similar
flexibility... so having a common "QPID" format (in XML) still makes sense
to me. However, certainly for the client I think it is essential we let the
use of the library decide where they source their configuration from (they
themselves may not have much choice).
>
> > A couple of quick questions...
> >
> > With the Java Client changes, will the client now always require a
> > configuration file? [ I hope not :-) ]
>
> The client would only require a config file if you wanted to configure
> options
> which would require a config file :-) Right now, this would mean SSL, but
> it
> could mean other options down the road. It should still be possible to get
> the
> client running without a config file.
Again - I think it is essential that the client can run without
configuration (pretty much as it does now)... we want to make it as easy as
possible to just drop in the appropriate jars and start using, without
having to worry about a myriad of config files (I'm starting to sound like
Alan)
... And at what level are you
> > planning to make these options configurable (application instance or
> > connection being the two most obvious).
>
> I haven't dug into the client code yet so I can't really say at what level
> is
> the most practical. Now that we've got the XML/YAML thing put to bed, I'll
> be
> looking at this shortly. I'd like to add it at the most flexible point
> possible
> (connection), but that will depend on what the code looks like.
Yes... thinking about it, the most obvious thing is that you look up
connection strings by name (in a config file, through JNDI or whatever).
You also then want to get the configure options for that connection (e.g. if
your connection is called MY_CONNECTION then you might want to get a config
value option by looking up MY_CONNECTION.<config option name> with further
levels of nesting if necessary).
-- Rob
Re: YAML Configuration (Was Re: QPID-6)
Posted by Kevin Smith <ks...@redhat.com>.
Kevin Smith wrote:
> Robert Greig wrote:
>> On 26/01/07, Tomas Restrepo <to...@devdeo.com> wrote:
>>
>>> It's OK for a broker to depend on explicit configuration, but a client
>>> library is usually used in very different contexts, and at least in
>>> some of
>>> those forcing external configuration is going to cause a lot of
>>> troubles.
>>> Likewise, it's very OK to be able to use external configuration (an
>>> XML file
>>> for example) as an *alternative* to the API, but shouldn't be the
>>> only way
>>> to set options like this, I think.
>>
>> Yes, agree completely.
>>
>> RG
>>
> OK. I feel like I haven't been clear about my intentions wrt the client.
> Let me outline what I've gotten written for the client:
>
> 1) org.apache.qpid.client.SSLConfiguration class which contains three
> pieces of info: keystore file path, keystore password, and cert type
> (defaults to "SunX509").
>
> 2) Modifications to the constructors on
> org.apache.qpid.client.AMQConnection to accept an additional
> SSLConfiguration argument.
>
> 3) Changed the logic in org.apache.qpid.client.AMQProtocolHandler to get
> the SSLConfiguration from it's parent AMQConnection and call the
> SSLContextFactory (from qpid commons) to create a SSLContext to pass to
> Mina.
>
> So, from a coding perspective the only thing which a developer needs to
> do to enable SSL is create an instance of SSLConfiguration with the
> appropriate information and then use that instance when creating a
> AMQConnection instance directly or via a AMQConnectionFactory.
>
> I still have some cleanup to do with FailoverHandler and write a few
> tests, but the basic work is done (I think). I think this satisfies the
> requirements I've heard on the list.
>
> --Kevin
>
I also wanted to mention that the code changes for the broker and commons are
attached as a patch to the JIRA issue, if anyone wants to take a look at that.
--Kevin
Re: YAML Configuration (Was Re: QPID-6)
Posted by Carl Trieloff <cc...@redhat.com>.
Kevin,
I think what you describe seems good. I am not sure we even need to add
any config as these options can easily be set pragmatically. I would be
inclined to do examples and leave it to the user to decide how to store
the options as config if they want to.
Carl.
Kevin Smith wrote:
> Robert Greig wrote:
>> On 26/01/07, Tomas Restrepo <to...@devdeo.com> wrote:
>>
>>> It's OK for a broker to depend on explicit configuration, but a client
>>> library is usually used in very different contexts, and at least in
>>> some of
>>> those forcing external configuration is going to cause a lot of
>>> troubles.
>>> Likewise, it's very OK to be able to use external configuration (an
>>> XML file
>>> for example) as an *alternative* to the API, but shouldn't be the
>>> only way
>>> to set options like this, I think.
>>
>> Yes, agree completely.
>>
>> RG
>>
> OK. I feel like I haven't been clear about my intentions wrt the
> client. Let me outline what I've gotten written for the client:
>
> 1) org.apache.qpid.client.SSLConfiguration class which contains three
> pieces of info: keystore file path, keystore password, and cert type
> (defaults to "SunX509").
>
> 2) Modifications to the constructors on
> org.apache.qpid.client.AMQConnection to accept an additional
> SSLConfiguration argument.
>
> 3) Changed the logic in org.apache.qpid.client.AMQProtocolHandler to
> get the SSLConfiguration from it's parent AMQConnection and call the
> SSLContextFactory (from qpid commons) to create a SSLContext to pass
> to Mina.
>
> So, from a coding perspective the only thing which a developer needs
> to do to enable SSL is create an instance of SSLConfiguration with the
> appropriate information and then use that instance when creating a
> AMQConnection instance directly or via a AMQConnectionFactory.
>
> I still have some cleanup to do with FailoverHandler and write a few
> tests, but the basic work is done (I think). I think this satisfies
> the requirements I've heard on the list.
>
> --Kevin
Re: YAML Configuration (Was Re: QPID-6)
Posted by Kevin Smith <ks...@redhat.com>.
Robert Greig wrote:
> On 26/01/07, Tomas Restrepo <to...@devdeo.com> wrote:
>
>> It's OK for a broker to depend on explicit configuration, but a client
>> library is usually used in very different contexts, and at least in
>> some of
>> those forcing external configuration is going to cause a lot of troubles.
>> Likewise, it's very OK to be able to use external configuration (an
>> XML file
>> for example) as an *alternative* to the API, but shouldn't be the only
>> way
>> to set options like this, I think.
>
> Yes, agree completely.
>
> RG
>
OK. I feel like I haven't been clear about my intentions wrt the client. Let me
outline what I've gotten written for the client:
1) org.apache.qpid.client.SSLConfiguration class which contains three pieces of
info: keystore file path, keystore password, and cert type (defaults to "SunX509").
2) Modifications to the constructors on org.apache.qpid.client.AMQConnection to
accept an additional SSLConfiguration argument.
3) Changed the logic in org.apache.qpid.client.AMQProtocolHandler to get the
SSLConfiguration from it's parent AMQConnection and call the SSLContextFactory
(from qpid commons) to create a SSLContext to pass to Mina.
So, from a coding perspective the only thing which a developer needs to do to
enable SSL is create an instance of SSLConfiguration with the appropriate
information and then use that instance when creating a AMQConnection instance
directly or via a AMQConnectionFactory.
I still have some cleanup to do with FailoverHandler and write a few tests, but
the basic work is done (I think). I think this satisfies the requirements I've
heard on the list.
--Kevin
Re: YAML Configuration (Was Re: QPID-6)
Posted by Robert Greig <ro...@gmail.com>.
On 26/01/07, Tomas Restrepo <to...@devdeo.com> wrote:
> It's OK for a broker to depend on explicit configuration, but a client
> library is usually used in very different contexts, and at least in some of
> those forcing external configuration is going to cause a lot of troubles.
> Likewise, it's very OK to be able to use external configuration (an XML file
> for example) as an *alternative* to the API, but shouldn't be the only way
> to set options like this, I think.
Yes, agree completely.
RG
Re: YAML Configuration (Was Re: QPID-6)
Posted by John O'Hara <jo...@gmail.com>.
How do OTHER Apache projects manage keyfiles and encryption bits and bobs?
We're not the first, I'm sure.
Also, at Apache there seems to be a strong tradition of XML style config
files -- even the style of the original httpd.conf file.
Let's keep with the Apache culture.
We should look at hot Tomcat and HTTPD and the Directory Project manage
client and server SSL settings...... who ever is interested in coding this
should #include one of those.
John
On 26/01/07, Colin Crist <co...@hermesjms.com> wrote:
>
>
> > >
> > > A couple of quick questions...
> > >
> > > With the Java Client changes, will the client now always require a
> > > configuration file? [ I hope not :-) ]
> >
> > The client would only require a config file if you wanted to
> > configure options which would require a config file :-) Right
> > now, this would mean SSL, but it could mean other options
> > down the road. It should still be possible to get the client
> > running without a config file.
>
> Applications, especailly in house ones, have enough configuration files as
> it is without requiring them to have yet another one for a messaging
> product.
>
> I personally try and get all my configuration into an RDBMS (if I've got
> one
> already) for example and avoid files but that does not work for all. I
> certainly don't mind files for server side config but the client is part
> of
> a bigger picture the less you dictate deployment to the user the better.
>
> What particular bit of SSL needs reference to a file? Can I not also just
> use strings as well in the ConnectionFactory so you get files and other
> sources of config? Files are fine for som3, but its good to give options
> for
> others.
>
> Finally, if I bind a connection factory into JNDI then files don't work.
>
> Regards,
>
> Colin.
>
>
>
>
Re: YAML Configuration (Was Re: QPID-6)
Posted by Kevin Smith <ks...@redhat.com>.
Tomas Restrepo wrote:
> Hi Kevin,
>
>>> With the Java Client changes, will the client now always require a
>>> configuration file? [ I hope not :-) ]
>> The client would only require a config file if you wanted to configure
>> options
>> which would require a config file :-)
>
> I can't speak about the Java client per se since I don't know about how
> people expect to use it. However, I'm collaborating a bit on the .NET client
> and wondering a lot about some of the same issues you're talking about
> (including the SSL support which hasn't been added yet).
>
> Given this, I know how I'd like to use the .NET client for some stuff and
> things like SSL should definitely be configurable from code without needing
> an external configuration file, at least from in my very humble opinion. I
> would think that at the very least, the option to enable SSL or not as well
> as the client certificate should be easily exposed programmatically at the
> time the connection to the broker is opened, and I was thinking that
> exposing this in some way in the ConnectionInfo sounded like the a
> reasonable approach.
>
> It's OK for a broker to depend on explicit configuration, but a client
> library is usually used in very different contexts, and at least in some of
> those forcing external configuration is going to cause a lot of troubles.
> Likewise, it's very OK to be able to use external configuration (an XML file
> for example) as an *alternative* to the API, but shouldn't be the only way
> to set options like this, I think.
+1. This is what I was thinking. The approach I took with the broker was to add
API to allow it to be configured and then added the logic in the config layer to
call the appropriate API with the values from the file.
I was going to do something very similar for the client.
>
> Then again, it's just my opinion :)
>
> Tomas Restrepo
> tomas.restrepo@devdeo.com
> http://www.winterdom.com/weblog/
>
>
>
>
RE: YAML Configuration (Was Re: QPID-6)
Posted by Tomas Restrepo <to...@devdeo.com>.
Hi Kevin,
> > With the Java Client changes, will the client now always require a
> > configuration file? [ I hope not :-) ]
>
> The client would only require a config file if you wanted to configure
> options
> which would require a config file :-)
I can't speak about the Java client per se since I don't know about how
people expect to use it. However, I'm collaborating a bit on the .NET client
and wondering a lot about some of the same issues you're talking about
(including the SSL support which hasn't been added yet).
Given this, I know how I'd like to use the .NET client for some stuff and
things like SSL should definitely be configurable from code without needing
an external configuration file, at least from in my very humble opinion. I
would think that at the very least, the option to enable SSL or not as well
as the client certificate should be easily exposed programmatically at the
time the connection to the broker is opened, and I was thinking that
exposing this in some way in the ConnectionInfo sounded like the a
reasonable approach.
It's OK for a broker to depend on explicit configuration, but a client
library is usually used in very different contexts, and at least in some of
those forcing external configuration is going to cause a lot of troubles.
Likewise, it's very OK to be able to use external configuration (an XML file
for example) as an *alternative* to the API, but shouldn't be the only way
to set options like this, I think.
Then again, it's just my opinion :)
Tomas Restrepo
tomas.restrepo@devdeo.com
http://www.winterdom.com/weblog/
RE: YAML Configuration (Was Re: QPID-6)
Posted by Colin Crist <co...@hermesjms.com>.
> >
> > A couple of quick questions...
> >
> > With the Java Client changes, will the client now always require a
> > configuration file? [ I hope not :-) ]
>
> The client would only require a config file if you wanted to
> configure options which would require a config file :-) Right
> now, this would mean SSL, but it could mean other options
> down the road. It should still be possible to get the client
> running without a config file.
Applications, especailly in house ones, have enough configuration files as
it is without requiring them to have yet another one for a messaging
product.
I personally try and get all my configuration into an RDBMS (if I've got one
already) for example and avoid files but that does not work for all. I
certainly don't mind files for server side config but the client is part of
a bigger picture the less you dictate deployment to the user the better.
What particular bit of SSL needs reference to a file? Can I not also just
use strings as well in the ConnectionFactory so you get files and other
sources of config? Files are fine for som3, but its good to give options for
others.
Finally, if I bind a connection factory into JNDI then files don't work.
Regards,
Colin.
Re: YAML Configuration (Was Re: QPID-6)
Posted by Kevin Smith <ks...@redhat.com>.
Robert Godfrey wrote:
> Hi Kevin,
>
>> I've already attached a patch to the JIRA for the Java broker changes.
>> Once we
>> > can agree on one or the other I'll be able to finish the Java client
>> work.
>>
>> I think a (simple!) XML configuration file will do the trick and fully
>> support that.
>
>
> Given that we cannot control the context in which the client library is
> deployed, I think we need to be flexible in the way in which these
> configuration options can be set. That is I think that it should be
> possible to configure the library by passing in a
> commons.Configurationobject. In that sense for the (Java) client I
> think the decision over XML
> vs YAML vs Properties file etc may be something of a red-herring.
> However I
> am +1 for using XML as the default and as the basis for a common
> configuration style between clients / brokers of all languages.
This is an interesting idea. We could do that for both the broker and the
client, assuming that there's a commons-configuration class which supports the
user's desired format.
>
> A couple of quick questions...
>
> With the Java Client changes, will the client now always require a
> configuration file? [ I hope not :-) ]
The client would only require a config file if you wanted to configure options
which would require a config file :-) Right now, this would mean SSL, but it
could mean other options down the road. It should still be possible to get the
client running without a config file.
... And at what level are you
> planning to make these options configurable (application instance or
> connection being the two most obvious).
I haven't dug into the client code yet so I can't really say at what level is
the most practical. Now that we've got the XML/YAML thing put to bed, I'll be
looking at this shortly. I'd like to add it at the most flexible point possible
(connection), but that will depend on what the code looks like.
--Kevin
>
> Cheers,
> Rob
>
Re: YAML Configuration (Was Re: QPID-6)
Posted by Robert Godfrey <ro...@gmail.com>.
Hi Kevin,
> I've already attached a patch to the JIRA for the Java broker changes.
> Once we
> > can agree on one or the other I'll be able to finish the Java client
> work.
>
> I think a (simple!) XML configuration file will do the trick and fully
> support that.
Given that we cannot control the context in which the client library is
deployed, I think we need to be flexible in the way in which these
configuration options can be set. That is I think that it should be
possible to configure the library by passing in a
commons.Configurationobject. In that sense for the (Java) client I
think the decision over XML
vs YAML vs Properties file etc may be something of a red-herring. However I
am +1 for using XML as the default and as the basis for a common
configuration style between clients / brokers of all languages.
A couple of quick questions...
With the Java Client changes, will the client now always require a
configuration file? [ I hope not :-) ] ... And at what level are you
planning to make these options configurable (application instance or
connection being the two most obvious).
Cheers,
Rob
Re: YAML Configuration (Was Re: QPID-6)
Posted by Robert Greig <ro...@gmail.com>.
On 26/01/07, Kevin Smith <ks...@redhat.com> wrote:
> If I had really, really had to pick, I'd pick XML for two reasons: 1) we've
> already got XML config file support built in for the Java broker, and 2) people
> are generally more familiar with XML than YAML.
I agree with this.
> I think we should pick one config file formatting language and use it
> everywhere. Having YAML for the Java pieces and XML for C++ pieces (would this
> also include the Python and Ruby clients?) is not where I think we want to be.
Yes, agree also.
> I've already attached a patch to the JIRA for the Java broker changes. Once we
> can agree on one or the other I'll be able to finish the Java client work.
I think a (simple!) XML configuration file will do the trick and fully
support that.
RG
Re: YAML Configuration (Was Re: QPID-6)
Posted by Carl Trieloff <cc...@redhat.com>.
I think it would be safe to stick with XML. I would support that.
Carl.
Kevin Smith wrote:
> Robert Greig wrote:
>> On 26/01/07, Kevin Smith <ks...@redhat.com> wrote:
>>> I'm planning on writing this using pure YAML for
>>> commons-configuration soonish
>>> (over the weekend). My plan is to get this into
>>> commons-configuration proper
>>> since they already have a bug open on this
>>> (http://issues.apache.org/jira/browse/CONFIGURATION-201).
>>
>> I've looked at YAML briefly and we would still use xml for our java
>> broker deployments. Operate (i.e. support) teams are familiar with xml
>> and understand xml. I appreciate that may seem an odd comment to
>> people who have not worked extensively with operate teams. The yaml
>> native syntax is not exactly intuitive.
>>
>> The config file format is not for our benefit but that of our users.
>>
>> RG
>>
> So, I guess it boils down to do we want to use YAML or XML, then. My
> own goal is to address the obstacle which is currently blocking me on
> implementing SSL configuration, namely no configuration file
> infrastructure inside the Java client. Whether that config file is in
> YAML or XML doesn't matter that much to me.
>
> If I had really, really had to pick, I'd pick XML for two reasons: 1)
> we've already got XML config file support built in for the Java
> broker, and 2) people are generally more familiar with XML than YAML.
>
> I think we should pick one config file formatting language and use it
> everywhere. Having YAML for the Java pieces and XML for C++ pieces
> (would this also include the Python and Ruby clients?) is not where I
> think we want to be.
>
> I've already attached a patch to the JIRA for the Java broker changes.
> Once we can agree on one or the other I'll be able to finish the Java
> client work.
>
> --Kevin
Re: YAML Configuration (Was Re: QPID-6)
Posted by Kevin Smith <ks...@redhat.com>.
Robert Greig wrote:
> On 26/01/07, Kevin Smith <ks...@redhat.com> wrote:
>> I'm planning on writing this using pure YAML for commons-configuration
>> soonish
>> (over the weekend). My plan is to get this into commons-configuration
>> proper
>> since they already have a bug open on this
>> (http://issues.apache.org/jira/browse/CONFIGURATION-201).
>
> I've looked at YAML briefly and we would still use xml for our java
> broker deployments. Operate (i.e. support) teams are familiar with xml
> and understand xml. I appreciate that may seem an odd comment to
> people who have not worked extensively with operate teams. The yaml
> native syntax is not exactly intuitive.
>
> The config file format is not for our benefit but that of our users.
>
> RG
>
So, I guess it boils down to do we want to use YAML or XML, then. My own goal is
to address the obstacle which is currently blocking me on implementing SSL
configuration, namely no configuration file infrastructure inside the Java
client. Whether that config file is in YAML or XML doesn't matter that much to me.
If I had really, really had to pick, I'd pick XML for two reasons: 1) we've
already got XML config file support built in for the Java broker, and 2) people
are generally more familiar with XML than YAML.
I think we should pick one config file formatting language and use it
everywhere. Having YAML for the Java pieces and XML for C++ pieces (would this
also include the Python and Ruby clients?) is not where I think we want to be.
I've already attached a patch to the JIRA for the Java broker changes. Once we
can agree on one or the other I'll be able to finish the Java client work.
--Kevin
Re: YAML Configuration (Was Re: QPID-6)
Posted by Robert Greig <ro...@gmail.com>.
On 26/01/07, Kevin Smith <ks...@redhat.com> wrote:
> I'm planning on writing this using pure YAML for commons-configuration soonish
> (over the weekend). My plan is to get this into commons-configuration proper
> since they already have a bug open on this
> (http://issues.apache.org/jira/browse/CONFIGURATION-201).
I've looked at YAML briefly and we would still use xml for our java
broker deployments. Operate (i.e. support) teams are familiar with xml
and understand xml. I appreciate that may seem an odd comment to
people who have not worked extensively with operate teams. The yaml
native syntax is not exactly intuitive.
The config file format is not for our benefit but that of our users.
RG