You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Eric Evans <ee...@acunu.com> on 2011/09/06 19:20:59 UTC
Proposal: Moving CQL drivers
So following on from the previous discussion[1] about moving the CQL
drivers out of tree, my interpretation of that discussion is that we
have consensus that this should be done, and that they should go in
Google Code and Apache Extras. Further, it seems a foregone
conclusion that Git be used, and so I also assume that's a safe
assumption, at least for the two existing drivers (Python and Java).
One of the obstacles was the tight coupling of components that the
JDBC driver depends on, which should now be solved in trunk
(https://issues.apache.org/jira/browse/CASSANDRA-2936). Other than
that the only items remaining relate to updating the tests so that
they can connect to an already running instance, instead of the
setup/teardown of nodes that is done now.
Here is what I propose to do (one driver at a time):
1. Setup new Git-based projects on Apache Extras (cassandra-jdbc and
cassandra-dbapi2)
2. Import the most current code and fix-up the tests as needed
3. Submit issues w/ patches for the removal of the corresponding
driver from Cassandra
I would like to have this in place for 1.0, and believe it should be
OK despite the fact that we are freezing on September 8 (we're still
freezing on 9/8 I assume?). Since the plan was to delete the drivers
directory after branching anyway, this shouldn't be disruptive to the
release (even if drivers/ weren't deleted, the only functional change
would be to remove JDBC build and test targets from build.xml).
Thoughts?
[1]: http://thread.gmane.org/gmane.comp.db.cassandra.devel/4075
--
Eric Evans
Acunu | http://www.acunu.com | @acunu
Re: Proposal: Moving CQL drivers
Posted by Eric Evans <ee...@acunu.com>.
On Tue, Sep 6, 2011 at 12:20 PM, Eric Evans <ee...@acunu.com> wrote:
> Here is what I propose to do (one driver at a time):
>
> 1. Setup new Git-based projects on Apache Extras (cassandra-jdbc and
> cassandra-dbapi2)
> 2. Import the most current code and fix-up the tests as needed
> 3. Submit issues w/ patches for the removal of the corresponding
> driver from Cassandra
To update this, the Python driver should be ready to go. I've create
an Apache Extras project, and imported the code (including an revised
copy of test_cql.py).
The JIRA issue for this is
https://issues.apache.org/jira/browse/CASSANDRA-3180. Once it has
passed review (help with this appreciated) we can remove the driver
from trunk and proceed to the JDBC driver.
Thanks.
--
Eric Evans
Acunu | http://www.acunu.com | @acunu
Re: Proposal: Moving CQL drivers
Posted by Eric Evans <ee...@acunu.com>.
On Fri, Sep 9, 2011 at 3:32 PM, Nick Telford <ni...@gmail.com> wrote:
> It may be prudent to have someone to accept responsibility for each of the
> official drivers in Extras. This way we could have a single ticket (perhaps
> with sub-tickets for each driver) to ensure drivers are brought up to date
> for each release.
>
> This would be less about polluting the C* JIRA with implementation details
> for each driver, more of a place for driver maintainers to report-in that a
> driver is compatible with an impending release.
>
> This would also make it easier to identify if/when a driver is no longer
> being maintained, allowing us to strike it off the list of "official
> drivers" until it's been reined in.
>
> I guess it depends on the level to which we want the drivers associated with
> C*.
People come and go for various reasons so I'm not sure you could make
this work effectively. I do however like the idea of having some sort
of gate, or acceptance criteria, for being included in the list of
drivers that we associate ourselves with.
Perhaps we could devise a basic set of requirements for tests, and
then require that a common CI system somewhere be running the tests.
The test results could be used to determine (continued )acceptance.
--
Eric Evans
Acunu | http://www.acunu.com | @acunu
Re: Proposal: Moving CQL drivers
Posted by Nick Telford <ni...@gmail.com>.
It may be prudent to have someone to accept responsibility for each of the
official drivers in Extras. This way we could have a single ticket (perhaps
with sub-tickets for each driver) to ensure drivers are brought up to date
for each release.
This would be less about polluting the C* JIRA with implementation details
for each driver, more of a place for driver maintainers to report-in that a
driver is compatible with an impending release.
This would also make it easier to identify if/when a driver is no longer
being maintained, allowing us to strike it off the list of "official
drivers" until it's been reined in.
I guess it depends on the level to which we want the drivers associated with
C*.
Regards,
On 9 September 2011 15:21, Jonathan Ellis <jb...@gmail.com> wrote:
> On Fri, Sep 9, 2011 at 8:40 AM, Rick Shaw <wf...@gmail.com> wrote:
> > I worry that if the issue management is moved from the main JIRA for C*
> > sponsored drivers there will be synergy and awareness lost between client
> > and server. Did the Hector and Pelops folks see it as a good thing to
> have
> > their own?
>
> I can't speak for them, but I certainly see it as a good thing that
> Hector and Pelops and pycassa et al have their own trackers. It's bad
> enough having JDBC tickets tagged "affects C* 0.8.4" when they are in
> the same tree, it makes even less sense when they are elsewhere.
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>
Re: Proposal: Moving CQL drivers
Posted by Jonathan Ellis <jb...@gmail.com>.
On Fri, Sep 9, 2011 at 8:40 AM, Rick Shaw <wf...@gmail.com> wrote:
> I worry that if the issue management is moved from the main JIRA for C*
> sponsored drivers there will be synergy and awareness lost between client
> and server. Did the Hector and Pelops folks see it as a good thing to have
> their own?
I can't speak for them, but I certainly see it as a good thing that
Hector and Pelops and pycassa et al have their own trackers. It's bad
enough having JDBC tickets tagged "affects C* 0.8.4" when they are in
the same tree, it makes even less sense when they are elsewhere.
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: Proposal: Moving CQL drivers
Posted by Eric Evans <ee...@acunu.com>.
On Fri, Sep 9, 2011 at 2:40 PM, Rick Shaw <wf...@gmail.com> wrote:
> I worry that if the issue management is moved from the main JIRA for C*
> sponsored drivers there will be synergy and awareness lost between client
> and server. Did the Hector and Pelops folks see it as a good thing to have
> their own?
It's a valid point, but I think I think it's easier to solve that if
it becomes a problem, than it is to try and map the workflows of many
projects onto a tool meant for one.
If it does become an issue then we can make it something to work on,
which could include technical solutions like linking bug trackers.
For example, think of the way that Hudson/Jenkins uses JIRA's SOAP API
(I assume) to update issues on success/failure.
> As a small convenience to the Maven inclined among us, perhaps it would be
> useful and non threatening to maintain the directory structure for the JDBC
> driver in default Maven directory structure format? This would be
> exceedingly helpful for simple source control checkout of the code base for
> those that would like to add their own POM to build and test, and not effect
> a default ANT build in any way. Yes I know it is possible to remap all the
> various directory resources but if it start out using the Maven defaults
> both camps can easily interoperate without a lot of special casing as it
> should not really matter to an ANT build design.
I have no personal objections, but I would ask that we put off things
like this until the move is complete.
--
Eric Evans
Acunu | http://www.acunu.com | @acunu
Re: Proposal: Moving CQL drivers
Posted by Rick Shaw <wf...@gmail.com>.
I worry that if the issue management is moved from the main JIRA for C*
sponsored drivers there will be synergy and awareness lost between client
and server. Did the Hector and Pelops folks see it as a good thing to have
their own?
As a small convenience to the Maven inclined among us, perhaps it would be
useful and non threatening to maintain the directory structure for the JDBC
driver in default Maven directory structure format? This would be
exceedingly helpful for simple source control checkout of the code base for
those that would like to add their own POM to build and test, and not effect
a default ANT build in any way. Yes I know it is possible to remap all the
various directory resources but if it start out using the Maven defaults
both camps can easily interoperate without a lot of special casing as it
should not really matter to an ANT build design.
On Wed, Sep 7, 2011 at 10:49 AM, Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:
> On 7 September 2011 10:12, Eric Evans <ee...@acunu.com> wrote:
> > On Wed, Sep 7, 2011 at 8:22 AM, Stephen Connolly
> > <st...@gmail.com> wrote:
> >> On 6 September 2011 18:34, Vivek Mishra <vi...@yahoo.com> wrote:
> >>> Sounds good moving to github.
> >>> 1 quick question, what about JIRAs already raised w.r.t drivers? Not
> sure but is it possible to integrate these new projects with current JIRA
> flow?
> >>>
> >>> Planning to make these new projects based on maven build process?(As
> that might be helpful in case of any quick release required for any sub
> module).
> >>
> >> Ha!
> >>
> >> I would be genuinely surprised if that were to happen.
> >>
> >> I think there is a greater chance of seeing C* itself being built with
> >> maven than the drivers...
> >
> > Really? I see the exact opposite, (and it's probably no secret how I
> > feel about Maven).
> >
>
> Hmmm, well here's my view, the only ones where Maven makes sense are
> the JVM based drivers. Most of the JVM based ones can be simplified
> down to the JDBC driver... and as Eric is the driver of the JDBC
> driver and Eric's opinions on Maven are well known...
>
> >> P.S.
> >> I will wait to be asked for my opinion on how this could be addresses
> >> using Maven as a build tool.
> >
> > Is that, "Through the use of copious amounts of XML markup, storage,
> > and network bandwidth?". :)
> >
>
> Meh, the XML would be an order of magnitude less than the current ANT
> build script.
>
> Storage... well it depends which storage you are talking about, it
> would be a choice of storage in .svn or storage in ~/.m2
>
> Network bandwidth... only if people follow poor -SNAPSHOT strategies
> and are constantly deploying -SNAPSHOTs to the remote repo....
>
> > Sorry, I couldn't resist.
>
> Neither could I ;-)
>
> >
> >> The stated preference of the C*
> >> developers is to use ANT. I am happy that Maven ANT Tasks is being
> >> used over IVY, and happy that the artifacts are being pushed to
> >> central, after that it doesn't matter what the build tool used is, as
> >> long as the published poms are good (and last time I fine tuned them
> >> they were) and as long as stuff gets into central, I am fine.
> >
> >
> >
> > --
> > Eric Evans
> > Acunu | http://www.acunu.com | @acunu
> >
>
Re: Proposal: Moving CQL drivers
Posted by Stephen Connolly <st...@gmail.com>.
On 7 September 2011 10:12, Eric Evans <ee...@acunu.com> wrote:
> On Wed, Sep 7, 2011 at 8:22 AM, Stephen Connolly
> <st...@gmail.com> wrote:
>> On 6 September 2011 18:34, Vivek Mishra <vi...@yahoo.com> wrote:
>>> Sounds good moving to github.
>>> 1 quick question, what about JIRAs already raised w.r.t drivers? Not sure but is it possible to integrate these new projects with current JIRA flow?
>>>
>>> Planning to make these new projects based on maven build process?(As that might be helpful in case of any quick release required for any sub module).
>>
>> Ha!
>>
>> I would be genuinely surprised if that were to happen.
>>
>> I think there is a greater chance of seeing C* itself being built with
>> maven than the drivers...
>
> Really? I see the exact opposite, (and it's probably no secret how I
> feel about Maven).
>
Hmmm, well here's my view, the only ones where Maven makes sense are
the JVM based drivers. Most of the JVM based ones can be simplified
down to the JDBC driver... and as Eric is the driver of the JDBC
driver and Eric's opinions on Maven are well known...
>> P.S.
>> I will wait to be asked for my opinion on how this could be addresses
>> using Maven as a build tool.
>
> Is that, "Through the use of copious amounts of XML markup, storage,
> and network bandwidth?". :)
>
Meh, the XML would be an order of magnitude less than the current ANT
build script.
Storage... well it depends which storage you are talking about, it
would be a choice of storage in .svn or storage in ~/.m2
Network bandwidth... only if people follow poor -SNAPSHOT strategies
and are constantly deploying -SNAPSHOTs to the remote repo....
> Sorry, I couldn't resist.
Neither could I ;-)
>
>> The stated preference of the C*
>> developers is to use ANT. I am happy that Maven ANT Tasks is being
>> used over IVY, and happy that the artifacts are being pushed to
>> central, after that it doesn't matter what the build tool used is, as
>> long as the published poms are good (and last time I fine tuned them
>> they were) and as long as stuff gets into central, I am fine.
>
>
>
> --
> Eric Evans
> Acunu | http://www.acunu.com | @acunu
>
Re: Proposal: Moving CQL drivers
Posted by Eric Evans <ee...@acunu.com>.
On Wed, Sep 7, 2011 at 8:22 AM, Stephen Connolly
<st...@gmail.com> wrote:
> On 6 September 2011 18:34, Vivek Mishra <vi...@yahoo.com> wrote:
>> Sounds good moving to github.
>> 1 quick question, what about JIRAs already raised w.r.t drivers? Not sure but is it possible to integrate these new projects with current JIRA flow?
>>
>> Planning to make these new projects based on maven build process?(As that might be helpful in case of any quick release required for any sub module).
>
> Ha!
>
> I would be genuinely surprised if that were to happen.
>
> I think there is a greater chance of seeing C* itself being built with
> maven than the drivers...
Really? I see the exact opposite, (and it's probably no secret how I
feel about Maven).
> P.S.
> I will wait to be asked for my opinion on how this could be addresses
> using Maven as a build tool.
Is that, "Through the use of copious amounts of XML markup, storage,
and network bandwidth?". :)
Sorry, I couldn't resist.
> The stated preference of the C*
> developers is to use ANT. I am happy that Maven ANT Tasks is being
> used over IVY, and happy that the artifacts are being pushed to
> central, after that it doesn't matter what the build tool used is, as
> long as the published poms are good (and last time I fine tuned them
> they were) and as long as stuff gets into central, I am fine.
--
Eric Evans
Acunu | http://www.acunu.com | @acunu
Re: Proposal: Moving CQL drivers
Posted by Stephen Connolly <st...@gmail.com>.
On 6 September 2011 18:34, Vivek Mishra <vi...@yahoo.com> wrote:
> Sounds good moving to github.
> 1 quick question, what about JIRAs already raised w.r.t drivers? Not sure but is it possible to integrate these new projects with current JIRA flow?
>
> Planning to make these new projects based on maven build process?(As that might be helpful in case of any quick release required for any sub module).
Ha!
I would be genuinely surprised if that were to happen.
I think there is a greater chance of seeing C* itself being built with
maven than the drivers...
-Stephen
P.S.
I will wait to be asked for my opinion on how this could be addresses
using Maven as a build tool. The stated preference of the C*
developers is to use ANT. I am happy that Maven ANT Tasks is being
used over IVY, and happy that the artifacts are being pushed to
central, after that it doesn't matter what the build tool used is, as
long as the published poms are good (and last time I fine tuned them
they were) and as long as stuff gets into central, I am fine.
>
> Any subsequent Cassandra release will be independent on driver release/s or vice versa?
>
> What about if creating a single github project like CQLDrivers and creating jdbc and dbapi2 as sub projects of it?
>
>
> As a developer it can help in case somebody needs to check in some stuff in both of these.
>
>
> -Vivek
>
>
>
>
> ________________________________
> From: Eric Evans <ee...@acunu.com>
> To: dev@cassandra.apache.org
> Sent: Tuesday, September 6, 2011 10:50 PM
> Subject: Proposal: Moving CQL drivers
>
> So following on from the previous discussion[1] about moving the CQL
> drivers out of tree, my interpretation of that discussion is that we
> have consensus that this should be done, and that they should go in
> Google Code and Apache Extras. Further, it seems a foregone
> conclusion that Git be used, and so I also assume that's a safe
> assumption, at least for the two existing drivers (Python and Java).
>
> One of the obstacles was the tight coupling of components that the
> JDBC driver depends on, which should now be solved in trunk
> (https://issues.apache.org/jira/browse/CASSANDRA-2936). Other than
> that the only items remaining relate to updating the tests so that
> they can connect to an already running instance, instead of the
> setup/teardown of nodes that is done now.
>
> Here is what I propose to do (one driver at a time):
>
> 1. Setup new Git-based projects on Apache Extras (cassandra-jdbc and
> cassandra-dbapi2)
> 2. Import the most current code and fix-up the tests as needed
> 3. Submit issues w/ patches for the removal of the corresponding
> driver from Cassandra
>
> I would like to have this in place for 1.0, and believe it should be
> OK despite the fact that we are freezing on September 8 (we're still
> freezing on 9/8 I assume?). Since the plan was to delete the drivers
> directory after branching anyway, this shouldn't be disruptive to the
> release (even if drivers/ weren't deleted, the only functional change
> would be to remove JDBC build and test targets from build.xml).
>
> Thoughts?
>
>
> [1]: http://thread.gmane.org/gmane.comp.db.cassandra.devel/4075
>
> --
> Eric Evans
> Acunu | http://www.acunu.com | @acunu
Re: Proposal: Moving CQL drivers
Posted by Eric Evans <ee...@acunu.com>.
On Tue, Sep 6, 2011 at 6:34 PM, Vivek Mishra <vi...@yahoo.com> wrote:
> Sounds good moving to github.
It's Google Code/Apache Extras that we've been discussing actually
(http://code.google.com/a/apache-extras.org/hosting).
> 1 quick question, what about JIRAs already raised w.r.t drivers? Not sure but is it possible to integrate these new projects with current JIRA flow?
I expect that they'll be moved to their respective bug trackers. I'm
not sure you'd want to integrate that with the Jira workflow since
that is something else that is pretty project-centric.
> Planning to make these new projects based on maven build process?(As that might be helpful in case of any quick release required for any sub module).
Though it pains me personally to say so, this is probably one
advantage to decoupling the drivers from Cassandra. If the
contributors to that particular project are less biased against Maven
then say, me, then it would be more likely to happen.
There must be something similar to Godwin's Law that states that as a
technical discussion grows longer, the probability that someone will
advocate Maven approaches one. :)
> Any subsequent Cassandra release will be independent on driver release/s or vice versa?
This was always the case, so yes.
> What about if creating a single github project like CQLDrivers and creating jdbc and dbapi2 as sub projects of it?
>
>
> As a developer it can help in case somebody needs to check in some stuff in both of these.
Consensus seemed to be that Apache Extras (Google Code) would be
better from a branding perspective, and I don't think you can
structure projects like that there.
That said, I'm sure there will be folks that contribute to more than
one, but I'm guessing that as the number of drivers and overall
contributors increase, those people will be the exception rather than
the rule (i.e. I'm not sure it makes sense to optimize for them).
--
Eric Evans
Acunu | http://www.acunu.com | @acunu
Re: Proposal: Moving CQL drivers
Posted by Vivek Mishra <vi...@yahoo.com>.
Sounds good moving to github.
1 quick question, what about JIRAs already raised w.r.t drivers? Not sure but is it possible to integrate these new projects with current JIRA flow?
Planning to make these new projects based on maven build process?(As that might be helpful in case of any quick release required for any sub module).
Any subsequent Cassandra release will be independent on driver release/s or vice versa?
What about if creating a single github project like CQLDrivers and creating jdbc and dbapi2 as sub projects of it?
As a developer it can help in case somebody needs to check in some stuff in both of these.
-Vivek
________________________________
From: Eric Evans <ee...@acunu.com>
To: dev@cassandra.apache.org
Sent: Tuesday, September 6, 2011 10:50 PM
Subject: Proposal: Moving CQL drivers
So following on from the previous discussion[1] about moving the CQL
drivers out of tree, my interpretation of that discussion is that we
have consensus that this should be done, and that they should go in
Google Code and Apache Extras. Further, it seems a foregone
conclusion that Git be used, and so I also assume that's a safe
assumption, at least for the two existing drivers (Python and Java).
One of the obstacles was the tight coupling of components that the
JDBC driver depends on, which should now be solved in trunk
(https://issues.apache.org/jira/browse/CASSANDRA-2936). Other than
that the only items remaining relate to updating the tests so that
they can connect to an already running instance, instead of the
setup/teardown of nodes that is done now.
Here is what I propose to do (one driver at a time):
1. Setup new Git-based projects on Apache Extras (cassandra-jdbc and
cassandra-dbapi2)
2. Import the most current code and fix-up the tests as needed
3. Submit issues w/ patches for the removal of the corresponding
driver from Cassandra
I would like to have this in place for 1.0, and believe it should be
OK despite the fact that we are freezing on September 8 (we're still
freezing on 9/8 I assume?). Since the plan was to delete the drivers
directory after branching anyway, this shouldn't be disruptive to the
release (even if drivers/ weren't deleted, the only functional change
would be to remove JDBC build and test targets from build.xml).
Thoughts?
[1]: http://thread.gmane.org/gmane.comp.db.cassandra.devel/4075
--
Eric Evans
Acunu | http://www.acunu.com | @acunu
Re: Proposal: Moving CQL drivers
Posted by Robert Jackson <ro...@promedicalinc.com>.
This looks like a good plan.
Once the drivers are moved to Google Code, would we be managing/tracking issues via Google Code or still via JIRA? I would assume Google Code, but wanted to clarify.
Since each driver will likely be in it's own repository I would also like to see the ruby driver [1] migrated as well. It doesn't necessarily make sense to merge it into the main C* tree just to move it out, and it does appear ready for publishing to me. At this point it is fully functional, and performs as well or better than the thrift-api based fauna/cassandra on some simple tests I performed [2].
[1] - https://issues.apache.org/jira/browse/CASSANDRA-2500
[2] - https://gist.github.com/1185026
Robert Jackson
----- Original Message -----
From: "Eric Evans" <ee...@acunu.com>
To: dev@cassandra.apache.org
Sent: Tuesday, September 6, 2011 1:20:59 PM
Subject: Proposal: Moving CQL drivers
So following on from the previous discussion[1] about moving the CQL
drivers out of tree, my interpretation of that discussion is that we
have consensus that this should be done, and that they should go in
Google Code and Apache Extras. Further, it seems a foregone
conclusion that Git be used, and so I also assume that's a safe
assumption, at least for the two existing drivers (Python and Java).
One of the obstacles was the tight coupling of components that the
JDBC driver depends on, which should now be solved in trunk
(https://issues.apache.org/jira/browse/CASSANDRA-2936). Other than
that the only items remaining relate to updating the tests so that
they can connect to an already running instance, instead of the
setup/teardown of nodes that is done now.
Here is what I propose to do (one driver at a time):
1. Setup new Git-based projects on Apache Extras (cassandra-jdbc and
cassandra-dbapi2)
2. Import the most current code and fix-up the tests as needed
3. Submit issues w/ patches for the removal of the corresponding
driver from Cassandra
I would like to have this in place for 1.0, and believe it should be
OK despite the fact that we are freezing on September 8 (we're still
freezing on 9/8 I assume?). Since the plan was to delete the drivers
directory after branching anyway, this shouldn't be disruptive to the
release (even if drivers/ weren't deleted, the only functional change
would be to remove JDBC build and test targets from build.xml).
Thoughts?
[1]: http://thread.gmane.org/gmane.comp.db.cassandra.devel/4075
--
Eric Evans
Acunu | http://www.acunu.com | @acunu