Re: [DISCUSS] Move Avatica to a sub-project?

[Julian Hyde <jh...@apache.org>]

My feeling is that Avatica ultimately needs to be a TLP, with its own governance. Calcite is, in a sense, incubating it until it is ready. My concern is that if we "release the pressure” we’ll lose the impetus to make it a TLP and get stuck half-way.

That said, Avatica does not have enough activity to be a TLP right now. So let’s start the separation, so that Avatica will be perceived as more independent.

I think Solr is a good example to follow. It is a sub-project of Lucene but is branded separately; for instance, it has its own site: http://lucene.apache.org/solr/ but the projects share a dev list. I like the idea of Avatica having its own site, say http://calcite.apache.org/avatica.

We will need to create a new site directory (in Jekyll layout), a new Avatica parent POM, and a new history.md. However, I am neutral on whether the git repos need to be separated at this point. 

Julian



> On Jan 30, 2016, at 8:47 PM, Ted Dunning <te...@gmail.com> wrote:
> 
> I would suggest just having a separate release artifact for a time before
> spinning out a separate TLP.  Separate TLP is a pain in the *.
> 
> Speaking from experience ages ago with Mahout, having a separate artifact
> that had a different audience than the main project worked just fine.
> 
> 
> On Fri, Jan 29, 2016 at 10:29 PM, Josh Elser <jo...@gmail.com> wrote:
> 
>> Yeah, sub project is probably not the right terminology in retrospect. I'm
>> not sure what the word for it is: I was suggesting just another repository
>> and everything else stays the same. Glad you knew what I meant to say.
>> 
>> Maybe the question right now is: what would be gained by having a separate
>> PMC (ignoring community building type questions)? I can envision Avatica
>> eventually being mature enough to be a TLP, but would it help to start
>> splitting things now while trying to grow involvement (and solve the
>> community size issues)? Is the middle step worth the effort?
>> On Jan 29, 2016 8:27 PM, "Julian Hyde" <jh...@apache.org> wrote:
>> 
>>> Allow me to play devil’s advocate and to look at some other options.
>>> 
>>> What would be the practical difference between a sub-project and what we
>>> have now?
>>> * The code split into different repositories
>>> * De-coupled release schedule
>>> * More distinct web site
>>> * But still the same namespace, org.apache.calcite
>>> 
>>> By the way, I don’t think what you are proposing is a sub-project in the
>>> Apache sense. (For example, Apache Derby is a sub-project of Apache DB.
>>> Derby’s PMC votes on releases, but DB’s PMC reports to the Apache
>> Board.) I
>>> gather that the Board is apparently no longer very fond of subprojects.
>>> 
>>> What you are proposing, I think, would be a module of Calcite (or two -
>>> avatica and avatica-server) whose release schedule is decoupled from the
>>> main project’s release schedule.
>>> 
>>> And let’s consider the other alternative: splitting Avatica out as a
>>> top-level project (as ORC recently did from Hive). If Avatica became a
>>> top-level project would naturally have its own repo, release schedule,
>> and
>>> could have its own web site and name space, org.apache.avatica. It would
>>> also have its own governance, i.e. a PMC that reports to the Board.
>>> 
>>> It seems to me that Avatica, the software, makes more sense as a
>> top-level
>>> project. Does it make sense for Avatica, the community? I think so. You
>> are
>>> using Avatica for Phoenix independent of Calcite, and others are doing
>>> similar things. The only place we fall short is our number of active
>>> members. We need 3 active PMC members to make a release, and we basically
>>> have 2 right now (you and me).
>>> 
>>> If we agree that a TLP is the best option in terms of governance and
>>> perception then we could make a push to recruit more Avatica committers
>> and
>>> PMC members.
>>> 
>>> Julian
>>> 
>>> 
>>> 
>>>> On Jan 29, 2016, at 12:35 PM, Josh Elser <jo...@gmail.com> wrote:
>>>> 
>>>> Hi all,
>>>> 
>>>> I remember the question about spinning out Avatica was brought up
>> around
>>> the time Calcite graduation to TLP was happening.
>>>> 
>>>> Back then, I think Avatica was too early to really benefit from this
>>> distinction. Lately, I keep finding myself thinking that it might be
>> time.
>>> Of note, features/improvements that have happened since:
>>>> 
>>>> * Wire compatibility across releases (protobuf provides the
>>> building-blocks)
>>>> * Much better docs
>>>> * Steady increase in custom Avatica clients (people creating their own
>>> client) [1] is the best OSS example I've come across
>>>> * Insight into the Avatica server w/o hacking the code: Logging and
>>> metrics (still WIP, but hopefully landing soon)
>>>> 
>>>> In other words, we've gotten much better at defining what is Avatica
>> and
>>> how to use it, with an emphasis in stability across releases. This is big
>>> because a split from calcite "core" would require a very firm statement
>> of
>>> compatibility as Avatica changes would not be directly noticed to break
>>> "core" (as they would now in the same repo).
>>>> 
>>>> What I think makes sense is to spin Avatica into its own repository,
>>> still under the Calcite PMC umbrella. In other words, the Calcite PMC
>> would
>>> be responsible for both "Calcite" releases and "Avatica" releases, and
>>> releases of the one don't require a release of the other (although they
>> may
>>> continue to coincide). I don't believe their is significant interest to
>>> justify spinning off Avatica into its own project (w/ governance), thus
>> the
>>> "sub-project" works well.
>>>> 
>>>> What do others think? Assuming we have release automation down,
>>> hopefully the doubled release work would not be a big concern. What have
>> I
>>> overlooked?
>>>> 
>>>> - Josh
>>>> 
>>>> [1] https://bitbucket.org/lalinsky/python-phoenixdb
>>> 
>>> 
>> 

Re: [DISCUSS] Move Avatica to a sub-project?

[Josh Elser <jo...@gmail.com>]

Right, same thing, different way of saying it (avatica would not 
inherit/depend on the calcite parent pom).

Thanks again.

Julian Hyde wrote:
> Calcite’s parent pom is in the root directory of the repo, and it makes sense to keep it there.
>
> So I think Avatica would remain in the avatica directory, avatica/pom.xml would become a parent POM, we’d move avatica-server to become avatica/server and we’d move avatica/src to avatica/core/src. Then we’d add NOTICE, LICENSE, README, site under avatica.
>
> And we’d remove
>
>    <module>avatica</module>
>    <module>avatica-server</module>
>
> from the calcite parent pom.xml, and make Avatica’s pom.xml inherit directly from the Apache parent pom.
>
> Julian
>
>> On Feb 2, 2016, at 11:05 AM, Josh Elser<jo...@gmail.com>  wrote:
>>
>> Any thoughts on how the layout/versioning should work? We could move Avatica into a top-level directory and have it separated from the "calcite" repository. So, to build Avatica, you'd have to `cd avatica&&  mvn<foo>`. The calcite modules would pull from your local repo or the configured remote repo(s). This would make cross-cutting changes a bit harder to verify, but it's probably a necessary evil for the eventual separation.
>>
>> I don't think a separate Git repo is required now, but those are pretty cheap as far as INFRA goes (to my understanding) which is why I had suggested it originally.
>>
>> When I get a moment, I'll make some uber/epic JIRA issue to track the stuff to do which we can pile on.
>>
>> Thanks for your input, Julian and Ted.
>>
>> Julian Hyde wrote:
>>> My feeling is that Avatica ultimately needs to be a TLP, with its own governance. Calcite is, in a sense, incubating it until it is ready. My concern is that if we "release the pressure” we’ll lose the impetus to make it a TLP and get stuck half-way.
>>>
>>> That said, Avatica does not have enough activity to be a TLP right now. So let’s start the separation, so that Avatica will be perceived as more independent.
>>>
>>> I think Solr is a good example to follow. It is a sub-project of Lucene but is branded separately; for instance, it has its own site: http://lucene.apache.org/solr/ but the projects share a dev list. I like the idea of Avatica having its own site, say http://calcite.apache.org/avatica.
>>>
>>> We will need to create a new site directory (in Jekyll layout), a new Avatica parent POM, and a new history.md. However, I am neutral on whether the git repos need to be separated at this point.
>>>
>>> Julian
>>>
>>>
>>>
>>>> On Jan 30, 2016, at 8:47 PM, Ted Dunning<te...@gmail.com>   wrote:
>>>>
>>>> I would suggest just having a separate release artifact for a time before
>>>> spinning out a separate TLP.  Separate TLP is a pain in the *.
>>>>
>>>> Speaking from experience ages ago with Mahout, having a separate artifact
>>>> that had a different audience than the main project worked just fine.
>>>>
>>>>
>>>> On Fri, Jan 29, 2016 at 10:29 PM, Josh Elser<jo...@gmail.com>   wrote:
>>>>
>>>>> Yeah, sub project is probably not the right terminology in retrospect. I'm
>>>>> not sure what the word for it is: I was suggesting just another repository
>>>>> and everything else stays the same. Glad you knew what I meant to say.
>>>>>
>>>>> Maybe the question right now is: what would be gained by having a separate
>>>>> PMC (ignoring community building type questions)? I can envision Avatica
>>>>> eventually being mature enough to be a TLP, but would it help to start
>>>>> splitting things now while trying to grow involvement (and solve the
>>>>> community size issues)? Is the middle step worth the effort?
>>>>> On Jan 29, 2016 8:27 PM, "Julian Hyde"<jh...@apache.org>   wrote:
>>>>>
>>>>>> Allow me to play devil’s advocate and to look at some other options.
>>>>>>
>>>>>> What would be the practical difference between a sub-project and what we
>>>>>> have now?
>>>>>> * The code split into different repositories
>>>>>> * De-coupled release schedule
>>>>>> * More distinct web site
>>>>>> * But still the same namespace, org.apache.calcite
>>>>>>
>>>>>> By the way, I don’t think what you are proposing is a sub-project in the
>>>>>> Apache sense. (For example, Apache Derby is a sub-project of Apache DB.
>>>>>> Derby’s PMC votes on releases, but DB’s PMC reports to the Apache
>>>>> Board.) I
>>>>>> gather that the Board is apparently no longer very fond of subprojects.
>>>>>>
>>>>>> What you are proposing, I think, would be a module of Calcite (or two -
>>>>>> avatica and avatica-server) whose release schedule is decoupled from the
>>>>>> main project’s release schedule.
>>>>>>
>>>>>> And let’s consider the other alternative: splitting Avatica out as a
>>>>>> top-level project (as ORC recently did from Hive). If Avatica became a
>>>>>> top-level project would naturally have its own repo, release schedule,
>>>>> and
>>>>>> could have its own web site and name space, org.apache.avatica. It would
>>>>>> also have its own governance, i.e. a PMC that reports to the Board.
>>>>>>
>>>>>> It seems to me that Avatica, the software, makes more sense as a
>>>>> top-level
>>>>>> project. Does it make sense for Avatica, the community? I think so. You
>>>>> are
>>>>>> using Avatica for Phoenix independent of Calcite, and others are doing
>>>>>> similar things. The only place we fall short is our number of active
>>>>>> members. We need 3 active PMC members to make a release, and we basically
>>>>>> have 2 right now (you and me).
>>>>>>
>>>>>> If we agree that a TLP is the best option in terms of governance and
>>>>>> perception then we could make a push to recruit more Avatica committers
>>>>> and
>>>>>> PMC members.
>>>>>>
>>>>>> Julian
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Jan 29, 2016, at 12:35 PM, Josh Elser<jo...@gmail.com>   wrote:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I remember the question about spinning out Avatica was brought up
>>>>> around
>>>>>> the time Calcite graduation to TLP was happening.
>>>>>>> Back then, I think Avatica was too early to really benefit from this
>>>>>> distinction. Lately, I keep finding myself thinking that it might be
>>>>> time.
>>>>>> Of note, features/improvements that have happened since:
>>>>>>> * Wire compatibility across releases (protobuf provides the
>>>>>> building-blocks)
>>>>>>> * Much better docs
>>>>>>> * Steady increase in custom Avatica clients (people creating their own
>>>>>> client) [1] is the best OSS example I've come across
>>>>>>> * Insight into the Avatica server w/o hacking the code: Logging and
>>>>>> metrics (still WIP, but hopefully landing soon)
>>>>>>> In other words, we've gotten much better at defining what is Avatica
>>>>> and
>>>>>> how to use it, with an emphasis in stability across releases. This is big
>>>>>> because a split from calcite "core" would require a very firm statement
>>>>> of
>>>>>> compatibility as Avatica changes would not be directly noticed to break
>>>>>> "core" (as they would now in the same repo).
>>>>>>> What I think makes sense is to spin Avatica into its own repository,
>>>>>> still under the Calcite PMC umbrella. In other words, the Calcite PMC
>>>>> would
>>>>>> be responsible for both "Calcite" releases and "Avatica" releases, and
>>>>>> releases of the one don't require a release of the other (although they
>>>>> may
>>>>>> continue to coincide). I don't believe their is significant interest to
>>>>>> justify spinning off Avatica into its own project (w/ governance), thus
>>>>> the
>>>>>> "sub-project" works well.
>>>>>>> What do others think? Assuming we have release automation down,
>>>>>> hopefully the doubled release work would not be a big concern. What have
>>>>> I
>>>>>> overlooked?
>>>>>>> - Josh
>>>>>>>
>>>>>>> [1] https://bitbucket.org/lalinsky/python-phoenixdb
>

Re: [DISCUSS] Move Avatica to a sub-project?

[Ron Wheeler <rw...@artifact-software.com>]

The main reason to have a separate repo would be to have a different set 
of committers.

Ron


On 02/02/2016 2:16 PM, Julian Hyde wrote:
> Calcite’s parent pom is in the root directory of the repo, and it makes sense to keep it there.
>
> So I think Avatica would remain in the avatica directory, avatica/pom.xml would become a parent POM, we’d move avatica-server to become avatica/server and we’d move avatica/src to avatica/core/src. Then we’d add NOTICE, LICENSE, README, site under avatica.
>
> And we’d remove
>
>    <module>avatica</module>
>    <module>avatica-server</module>
>
> from the calcite parent pom.xml, and make Avatica’s pom.xml inherit directly from the Apache parent pom.
>
> Julian
>
>> On Feb 2, 2016, at 11:05 AM, Josh Elser <jo...@gmail.com> wrote:
>>
>> Any thoughts on how the layout/versioning should work? We could move Avatica into a top-level directory and have it separated from the "calcite" repository. So, to build Avatica, you'd have to `cd avatica && mvn <foo>`. The calcite modules would pull from your local repo or the configured remote repo(s). This would make cross-cutting changes a bit harder to verify, but it's probably a necessary evil for the eventual separation.
>>
>> I don't think a separate Git repo is required now, but those are pretty cheap as far as INFRA goes (to my understanding) which is why I had suggested it originally.
>>
>> When I get a moment, I'll make some uber/epic JIRA issue to track the stuff to do which we can pile on.
>>
>> Thanks for your input, Julian and Ted.
>>
>> Julian Hyde wrote:
>>> My feeling is that Avatica ultimately needs to be a TLP, with its own governance. Calcite is, in a sense, incubating it until it is ready. My concern is that if we "release the pressure” we’ll lose the impetus to make it a TLP and get stuck half-way.
>>>
>>> That said, Avatica does not have enough activity to be a TLP right now. So let’s start the separation, so that Avatica will be perceived as more independent.
>>>
>>> I think Solr is a good example to follow. It is a sub-project of Lucene but is branded separately; for instance, it has its own site: http://lucene.apache.org/solr/ but the projects share a dev list. I like the idea of Avatica having its own site, say http://calcite.apache.org/avatica.
>>>
>>> We will need to create a new site directory (in Jekyll layout), a new Avatica parent POM, and a new history.md. However, I am neutral on whether the git repos need to be separated at this point.
>>>
>>> Julian
>>>
>>>
>>>
>>>> On Jan 30, 2016, at 8:47 PM, Ted Dunning<te...@gmail.com>  wrote:
>>>>
>>>> I would suggest just having a separate release artifact for a time before
>>>> spinning out a separate TLP.  Separate TLP is a pain in the *.
>>>>
>>>> Speaking from experience ages ago with Mahout, having a separate artifact
>>>> that had a different audience than the main project worked just fine.
>>>>
>>>>
>>>> On Fri, Jan 29, 2016 at 10:29 PM, Josh Elser<jo...@gmail.com>  wrote:
>>>>
>>>>> Yeah, sub project is probably not the right terminology in retrospect. I'm
>>>>> not sure what the word for it is: I was suggesting just another repository
>>>>> and everything else stays the same. Glad you knew what I meant to say.
>>>>>
>>>>> Maybe the question right now is: what would be gained by having a separate
>>>>> PMC (ignoring community building type questions)? I can envision Avatica
>>>>> eventually being mature enough to be a TLP, but would it help to start
>>>>> splitting things now while trying to grow involvement (and solve the
>>>>> community size issues)? Is the middle step worth the effort?
>>>>> On Jan 29, 2016 8:27 PM, "Julian Hyde"<jh...@apache.org>  wrote:
>>>>>
>>>>>> Allow me to play devil’s advocate and to look at some other options.
>>>>>>
>>>>>> What would be the practical difference between a sub-project and what we
>>>>>> have now?
>>>>>> * The code split into different repositories
>>>>>> * De-coupled release schedule
>>>>>> * More distinct web site
>>>>>> * But still the same namespace, org.apache.calcite
>>>>>>
>>>>>> By the way, I don’t think what you are proposing is a sub-project in the
>>>>>> Apache sense. (For example, Apache Derby is a sub-project of Apache DB.
>>>>>> Derby’s PMC votes on releases, but DB’s PMC reports to the Apache
>>>>> Board.) I
>>>>>> gather that the Board is apparently no longer very fond of subprojects.
>>>>>>
>>>>>> What you are proposing, I think, would be a module of Calcite (or two -
>>>>>> avatica and avatica-server) whose release schedule is decoupled from the
>>>>>> main project’s release schedule.
>>>>>>
>>>>>> And let’s consider the other alternative: splitting Avatica out as a
>>>>>> top-level project (as ORC recently did from Hive). If Avatica became a
>>>>>> top-level project would naturally have its own repo, release schedule,
>>>>> and
>>>>>> could have its own web site and name space, org.apache.avatica. It would
>>>>>> also have its own governance, i.e. a PMC that reports to the Board.
>>>>>>
>>>>>> It seems to me that Avatica, the software, makes more sense as a
>>>>> top-level
>>>>>> project. Does it make sense for Avatica, the community? I think so. You
>>>>> are
>>>>>> using Avatica for Phoenix independent of Calcite, and others are doing
>>>>>> similar things. The only place we fall short is our number of active
>>>>>> members. We need 3 active PMC members to make a release, and we basically
>>>>>> have 2 right now (you and me).
>>>>>>
>>>>>> If we agree that a TLP is the best option in terms of governance and
>>>>>> perception then we could make a push to recruit more Avatica committers
>>>>> and
>>>>>> PMC members.
>>>>>>
>>>>>> Julian
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Jan 29, 2016, at 12:35 PM, Josh Elser<jo...@gmail.com>  wrote:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I remember the question about spinning out Avatica was brought up
>>>>> around
>>>>>> the time Calcite graduation to TLP was happening.
>>>>>>> Back then, I think Avatica was too early to really benefit from this
>>>>>> distinction. Lately, I keep finding myself thinking that it might be
>>>>> time.
>>>>>> Of note, features/improvements that have happened since:
>>>>>>> * Wire compatibility across releases (protobuf provides the
>>>>>> building-blocks)
>>>>>>> * Much better docs
>>>>>>> * Steady increase in custom Avatica clients (people creating their own
>>>>>> client) [1] is the best OSS example I've come across
>>>>>>> * Insight into the Avatica server w/o hacking the code: Logging and
>>>>>> metrics (still WIP, but hopefully landing soon)
>>>>>>> In other words, we've gotten much better at defining what is Avatica
>>>>> and
>>>>>> how to use it, with an emphasis in stability across releases. This is big
>>>>>> because a split from calcite "core" would require a very firm statement
>>>>> of
>>>>>> compatibility as Avatica changes would not be directly noticed to break
>>>>>> "core" (as they would now in the same repo).
>>>>>>> What I think makes sense is to spin Avatica into its own repository,
>>>>>> still under the Calcite PMC umbrella. In other words, the Calcite PMC
>>>>> would
>>>>>> be responsible for both "Calcite" releases and "Avatica" releases, and
>>>>>> releases of the one don't require a release of the other (although they
>>>>> may
>>>>>> continue to coincide). I don't believe their is significant interest to
>>>>>> justify spinning off Avatica into its own project (w/ governance), thus
>>>>> the
>>>>>> "sub-project" works well.
>>>>>>> What do others think? Assuming we have release automation down,
>>>>>> hopefully the doubled release work would not be a big concern. What have
>>>>> I
>>>>>> overlooked?
>>>>>>> - Josh
>>>>>>>
>>>>>>> [1] https://bitbucket.org/lalinsky/python-phoenixdb
>


-- 
Ron Wheeler
President
Artifact Software Inc
email: rwheeler@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102

Re: [DISCUSS] Move Avatica to a sub-project?

[Julian Hyde <jh...@apache.org>]

Calcite’s parent pom is in the root directory of the repo, and it makes sense to keep it there.

So I think Avatica would remain in the avatica directory, avatica/pom.xml would become a parent POM, we’d move avatica-server to become avatica/server and we’d move avatica/src to avatica/core/src. Then we’d add NOTICE, LICENSE, README, site under avatica.

And we’d remove 

  <module>avatica</module>
  <module>avatica-server</module>

from the calcite parent pom.xml, and make Avatica’s pom.xml inherit directly from the Apache parent pom.

Julian

> On Feb 2, 2016, at 11:05 AM, Josh Elser <jo...@gmail.com> wrote:
> 
> Any thoughts on how the layout/versioning should work? We could move Avatica into a top-level directory and have it separated from the "calcite" repository. So, to build Avatica, you'd have to `cd avatica && mvn <foo>`. The calcite modules would pull from your local repo or the configured remote repo(s). This would make cross-cutting changes a bit harder to verify, but it's probably a necessary evil for the eventual separation.
> 
> I don't think a separate Git repo is required now, but those are pretty cheap as far as INFRA goes (to my understanding) which is why I had suggested it originally.
> 
> When I get a moment, I'll make some uber/epic JIRA issue to track the stuff to do which we can pile on.
> 
> Thanks for your input, Julian and Ted.
> 
> Julian Hyde wrote:
>> My feeling is that Avatica ultimately needs to be a TLP, with its own governance. Calcite is, in a sense, incubating it until it is ready. My concern is that if we "release the pressure” we’ll lose the impetus to make it a TLP and get stuck half-way.
>> 
>> That said, Avatica does not have enough activity to be a TLP right now. So let’s start the separation, so that Avatica will be perceived as more independent.
>> 
>> I think Solr is a good example to follow. It is a sub-project of Lucene but is branded separately; for instance, it has its own site: http://lucene.apache.org/solr/ but the projects share a dev list. I like the idea of Avatica having its own site, say http://calcite.apache.org/avatica.
>> 
>> We will need to create a new site directory (in Jekyll layout), a new Avatica parent POM, and a new history.md. However, I am neutral on whether the git repos need to be separated at this point.
>> 
>> Julian
>> 
>> 
>> 
>>> On Jan 30, 2016, at 8:47 PM, Ted Dunning<te...@gmail.com>  wrote:
>>> 
>>> I would suggest just having a separate release artifact for a time before
>>> spinning out a separate TLP.  Separate TLP is a pain in the *.
>>> 
>>> Speaking from experience ages ago with Mahout, having a separate artifact
>>> that had a different audience than the main project worked just fine.
>>> 
>>> 
>>> On Fri, Jan 29, 2016 at 10:29 PM, Josh Elser<jo...@gmail.com>  wrote:
>>> 
>>>> Yeah, sub project is probably not the right terminology in retrospect. I'm
>>>> not sure what the word for it is: I was suggesting just another repository
>>>> and everything else stays the same. Glad you knew what I meant to say.
>>>> 
>>>> Maybe the question right now is: what would be gained by having a separate
>>>> PMC (ignoring community building type questions)? I can envision Avatica
>>>> eventually being mature enough to be a TLP, but would it help to start
>>>> splitting things now while trying to grow involvement (and solve the
>>>> community size issues)? Is the middle step worth the effort?
>>>> On Jan 29, 2016 8:27 PM, "Julian Hyde"<jh...@apache.org>  wrote:
>>>> 
>>>>> Allow me to play devil’s advocate and to look at some other options.
>>>>> 
>>>>> What would be the practical difference between a sub-project and what we
>>>>> have now?
>>>>> * The code split into different repositories
>>>>> * De-coupled release schedule
>>>>> * More distinct web site
>>>>> * But still the same namespace, org.apache.calcite
>>>>> 
>>>>> By the way, I don’t think what you are proposing is a sub-project in the
>>>>> Apache sense. (For example, Apache Derby is a sub-project of Apache DB.
>>>>> Derby’s PMC votes on releases, but DB’s PMC reports to the Apache
>>>> Board.) I
>>>>> gather that the Board is apparently no longer very fond of subprojects.
>>>>> 
>>>>> What you are proposing, I think, would be a module of Calcite (or two -
>>>>> avatica and avatica-server) whose release schedule is decoupled from the
>>>>> main project’s release schedule.
>>>>> 
>>>>> And let’s consider the other alternative: splitting Avatica out as a
>>>>> top-level project (as ORC recently did from Hive). If Avatica became a
>>>>> top-level project would naturally have its own repo, release schedule,
>>>> and
>>>>> could have its own web site and name space, org.apache.avatica. It would
>>>>> also have its own governance, i.e. a PMC that reports to the Board.
>>>>> 
>>>>> It seems to me that Avatica, the software, makes more sense as a
>>>> top-level
>>>>> project. Does it make sense for Avatica, the community? I think so. You
>>>> are
>>>>> using Avatica for Phoenix independent of Calcite, and others are doing
>>>>> similar things. The only place we fall short is our number of active
>>>>> members. We need 3 active PMC members to make a release, and we basically
>>>>> have 2 right now (you and me).
>>>>> 
>>>>> If we agree that a TLP is the best option in terms of governance and
>>>>> perception then we could make a push to recruit more Avatica committers
>>>> and
>>>>> PMC members.
>>>>> 
>>>>> Julian
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Jan 29, 2016, at 12:35 PM, Josh Elser<jo...@gmail.com>  wrote:
>>>>>> 
>>>>>> Hi all,
>>>>>> 
>>>>>> I remember the question about spinning out Avatica was brought up
>>>> around
>>>>> the time Calcite graduation to TLP was happening.
>>>>>> Back then, I think Avatica was too early to really benefit from this
>>>>> distinction. Lately, I keep finding myself thinking that it might be
>>>> time.
>>>>> Of note, features/improvements that have happened since:
>>>>>> * Wire compatibility across releases (protobuf provides the
>>>>> building-blocks)
>>>>>> * Much better docs
>>>>>> * Steady increase in custom Avatica clients (people creating their own
>>>>> client) [1] is the best OSS example I've come across
>>>>>> * Insight into the Avatica server w/o hacking the code: Logging and
>>>>> metrics (still WIP, but hopefully landing soon)
>>>>>> In other words, we've gotten much better at defining what is Avatica
>>>> and
>>>>> how to use it, with an emphasis in stability across releases. This is big
>>>>> because a split from calcite "core" would require a very firm statement
>>>> of
>>>>> compatibility as Avatica changes would not be directly noticed to break
>>>>> "core" (as they would now in the same repo).
>>>>>> What I think makes sense is to spin Avatica into its own repository,
>>>>> still under the Calcite PMC umbrella. In other words, the Calcite PMC
>>>> would
>>>>> be responsible for both "Calcite" releases and "Avatica" releases, and
>>>>> releases of the one don't require a release of the other (although they
>>>> may
>>>>> continue to coincide). I don't believe their is significant interest to
>>>>> justify spinning off Avatica into its own project (w/ governance), thus
>>>> the
>>>>> "sub-project" works well.
>>>>>> What do others think? Assuming we have release automation down,
>>>>> hopefully the doubled release work would not be a big concern. What have
>>>> I
>>>>> overlooked?
>>>>>> - Josh
>>>>>> 
>>>>>> [1] https://bitbucket.org/lalinsky/python-phoenixdb
>>>>> 
>> 

Re: [DISCUSS] Move Avatica to a sub-project?

[Josh Elser <jo...@gmail.com>]

Any thoughts on how the layout/versioning should work? We could move 
Avatica into a top-level directory and have it separated from the 
"calcite" repository. So, to build Avatica, you'd have to `cd avatica && 
mvn <foo>`. The calcite modules would pull from your local repo or the 
configured remote repo(s). This would make cross-cutting changes a bit 
harder to verify, but it's probably a necessary evil for the eventual 
separation.

I don't think a separate Git repo is required now, but those are pretty 
cheap as far as INFRA goes (to my understanding) which is why I had 
suggested it originally.

When I get a moment, I'll make some uber/epic JIRA issue to track the 
stuff to do which we can pile on.

Thanks for your input, Julian and Ted.

Julian Hyde wrote:
> My feeling is that Avatica ultimately needs to be a TLP, with its own governance. Calcite is, in a sense, incubating it until it is ready. My concern is that if we "release the pressure” we’ll lose the impetus to make it a TLP and get stuck half-way.
>
> That said, Avatica does not have enough activity to be a TLP right now. So let’s start the separation, so that Avatica will be perceived as more independent.
>
> I think Solr is a good example to follow. It is a sub-project of Lucene but is branded separately; for instance, it has its own site: http://lucene.apache.org/solr/ but the projects share a dev list. I like the idea of Avatica having its own site, say http://calcite.apache.org/avatica.
>
> We will need to create a new site directory (in Jekyll layout), a new Avatica parent POM, and a new history.md. However, I am neutral on whether the git repos need to be separated at this point.
>
> Julian
>
>
>
>> On Jan 30, 2016, at 8:47 PM, Ted Dunning<te...@gmail.com>  wrote:
>>
>> I would suggest just having a separate release artifact for a time before
>> spinning out a separate TLP.  Separate TLP is a pain in the *.
>>
>> Speaking from experience ages ago with Mahout, having a separate artifact
>> that had a different audience than the main project worked just fine.
>>
>>
>> On Fri, Jan 29, 2016 at 10:29 PM, Josh Elser<jo...@gmail.com>  wrote:
>>
>>> Yeah, sub project is probably not the right terminology in retrospect. I'm
>>> not sure what the word for it is: I was suggesting just another repository
>>> and everything else stays the same. Glad you knew what I meant to say.
>>>
>>> Maybe the question right now is: what would be gained by having a separate
>>> PMC (ignoring community building type questions)? I can envision Avatica
>>> eventually being mature enough to be a TLP, but would it help to start
>>> splitting things now while trying to grow involvement (and solve the
>>> community size issues)? Is the middle step worth the effort?
>>> On Jan 29, 2016 8:27 PM, "Julian Hyde"<jh...@apache.org>  wrote:
>>>
>>>> Allow me to play devil’s advocate and to look at some other options.
>>>>
>>>> What would be the practical difference between a sub-project and what we
>>>> have now?
>>>> * The code split into different repositories
>>>> * De-coupled release schedule
>>>> * More distinct web site
>>>> * But still the same namespace, org.apache.calcite
>>>>
>>>> By the way, I don’t think what you are proposing is a sub-project in the
>>>> Apache sense. (For example, Apache Derby is a sub-project of Apache DB.
>>>> Derby’s PMC votes on releases, but DB’s PMC reports to the Apache
>>> Board.) I
>>>> gather that the Board is apparently no longer very fond of subprojects.
>>>>
>>>> What you are proposing, I think, would be a module of Calcite (or two -
>>>> avatica and avatica-server) whose release schedule is decoupled from the
>>>> main project’s release schedule.
>>>>
>>>> And let’s consider the other alternative: splitting Avatica out as a
>>>> top-level project (as ORC recently did from Hive). If Avatica became a
>>>> top-level project would naturally have its own repo, release schedule,
>>> and
>>>> could have its own web site and name space, org.apache.avatica. It would
>>>> also have its own governance, i.e. a PMC that reports to the Board.
>>>>
>>>> It seems to me that Avatica, the software, makes more sense as a
>>> top-level
>>>> project. Does it make sense for Avatica, the community? I think so. You
>>> are
>>>> using Avatica for Phoenix independent of Calcite, and others are doing
>>>> similar things. The only place we fall short is our number of active
>>>> members. We need 3 active PMC members to make a release, and we basically
>>>> have 2 right now (you and me).
>>>>
>>>> If we agree that a TLP is the best option in terms of governance and
>>>> perception then we could make a push to recruit more Avatica committers
>>> and
>>>> PMC members.
>>>>
>>>> Julian
>>>>
>>>>
>>>>
>>>>> On Jan 29, 2016, at 12:35 PM, Josh Elser<jo...@gmail.com>  wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I remember the question about spinning out Avatica was brought up
>>> around
>>>> the time Calcite graduation to TLP was happening.
>>>>> Back then, I think Avatica was too early to really benefit from this
>>>> distinction. Lately, I keep finding myself thinking that it might be
>>> time.
>>>> Of note, features/improvements that have happened since:
>>>>> * Wire compatibility across releases (protobuf provides the
>>>> building-blocks)
>>>>> * Much better docs
>>>>> * Steady increase in custom Avatica clients (people creating their own
>>>> client) [1] is the best OSS example I've come across
>>>>> * Insight into the Avatica server w/o hacking the code: Logging and
>>>> metrics (still WIP, but hopefully landing soon)
>>>>> In other words, we've gotten much better at defining what is Avatica
>>> and
>>>> how to use it, with an emphasis in stability across releases. This is big
>>>> because a split from calcite "core" would require a very firm statement
>>> of
>>>> compatibility as Avatica changes would not be directly noticed to break
>>>> "core" (as they would now in the same repo).
>>>>> What I think makes sense is to spin Avatica into its own repository,
>>>> still under the Calcite PMC umbrella. In other words, the Calcite PMC
>>> would
>>>> be responsible for both "Calcite" releases and "Avatica" releases, and
>>>> releases of the one don't require a release of the other (although they
>>> may
>>>> continue to coincide). I don't believe their is significant interest to
>>>> justify spinning off Avatica into its own project (w/ governance), thus
>>> the
>>>> "sub-project" works well.
>>>>> What do others think? Assuming we have release automation down,
>>>> hopefully the doubled release work would not be a big concern. What have
>>> I
>>>> overlooked?
>>>>> - Josh
>>>>>
>>>>> [1] https://bitbucket.org/lalinsky/python-phoenixdb
>>>>
>

Re: [DISCUSS] Move Avatica to a sub-project?

[Ted Dunning <te...@gmail.com>]

On Tue, Feb 2, 2016 at 10:52 AM, Julian Hyde <jh...@apache.org> wrote:

> I think Solr is a good example to follow. It is a sub-project of Lucene
> but is branded separately; for instance, it has its own site:
> http://lucene.apache.org/solr/ but the projects share a dev list. I like
> the idea of Avatica having its own site, say
> http://calcite.apache.org/avatica.
>
> We will need to create a new site directory (in Jekyll layout), a new
> Avatica parent POM, and a new history.md. However, I am neutral on
> whether the git repos need to be separated at this point.
>

Solr is an example of exactly the opposite. It used to be a TLP and merged
back into the mother-ship due to version matching sorts of arguments along
with overlap of committers and community.