[DISCUSS] Two dot o branch to Master

[Rod Simpson <ro...@rodsimpson.com>]

Usergriders,

As most of you know, we have been working on upgrading the entity manager and some other core changes.  This work has been taking place in the two-dot-o branch.  The good news is that the 2.0 system is much, much faster than 1.0.  Additionally, the use of ElasticSearch opens up a lot of avenues for us in terms of adding new query grammar.

We are now ready to move this work into master.  I am proposing that we put the current master into a branch called one-dot-o, then move the two-dot-o code into master.  Here are the pros and cons as I see them:

Pros:
This will bring the existing work (and there is a lot of work happening) to a more visible platform.  All changes will have to be brought in via PR and it will allow us to scrutinize the code and work being done more carefully.  It also validates the 2.0 codebase as the primary focus of the project. 


Cons:
We don’t currently have an upgrade path from 1.0 to 2.0.  This is something that we intend to build but it may take some time to accomplish (currently we are thinking it is about 2-3 months out, possibly sooner).  It also could make doing additional releases more difficult (e.g. can we release from a branch that isn’t master?)  We are ready to release 1.0.1 as we have had quite a few contributions, so if we have to release from master, then this release needs to happen before putting the 2.0 code in.


Thoughts?

Rod



-- 
Rod Simpson
@rockerston
rodsimpson.com

Re: [DISCUSS] Two dot o branch to Master

[Dave Johnson <sn...@gmail.com>]

Sorry, I hit sent too soon. My intention is to write up a summary of the changes in two-dot-o so that we all understand where things will be once the merge is done. I will send that shortly.

- Dave


> On Nov 21, 2014, at 7:44 AM, Dave Johnson <sn...@gmail.com> wrote:
> 
> +1 for merging two-dot-o to master.
> 
> New Core Persistence system added which includes these new modules:
> 
>    - Collections
>    - Graph
>    - Common
>    - Queue
>    
> 
> On Thu, Nov 20, 2014 at 1:23 AM, Todd Nine <todd.nine@gmail.com <ma...@gmail.com>> wrote:
> I also agree that we should move our 2.0 branch into master, since it is
> now the primary focus of the project.
> 
> I'm unsure what the official policy surrounding releases and branches are.
> IMHO a branch in git is a branch, mater or otherwise, tags to preserve the
> commit we release from are what is truly important.
> 
> If we do 1 or 2 more maintenance releases of 1.0 and we're clear with our
> tagging of those releases, I feel a "one-dot-o" branch would be sufficient.
> 
> Lewis and Jake, any guidance here?
> 
> Thanks,
> Todd
> 
> On Wed Nov 19 2014 at 5:05:48 PM Rod Simpson <rod@rodsimpson.com <ma...@rodsimpson.com>> wrote:
> 
> > Usergriders,
> >
> > As most of you know, we have been working on upgrading the entity manager
> > and some other core changes.  This work has been taking place in the
> > two-dot-o branch.  The good news is that the 2.0 system is much, much
> > faster than 1.0.  Additionally, the use of ElasticSearch opens up a lot of
> > avenues for us in terms of adding new query grammar.
> >
> > We are now ready to move this work into master.  I am proposing that we
> > put the current master into a branch called one-dot-o, then move the
> > two-dot-o code into master.  Here are the pros and cons as I see them:
> >
> > Pros:
> > This will bring the existing work (and there is a lot of work happening)
> > to a more visible platform.  All changes will have to be brought in via PR
> > and it will allow us to scrutinize the code and work being done more
> > carefully.  It also validates the 2.0 codebase as the primary focus of the
> > project.
> >
> >
> > Cons:
> > We don’t currently have an upgrade path from 1.0 to 2.0.  This is
> > something that we intend to build but it may take some time to accomplish
> > (currently we are thinking it is about 2-3 months out, possibly sooner).
> > It also could make doing additional releases more difficult (e.g. can we
> > release from a branch that isn’t master?)  We are ready to release 1.0.1 as
> > we have had quite a few contributions, so if we have to release from
> > master, then this release needs to happen before putting the 2.0 code in.
> >
> >
> > Thoughts?
> >
> > Rod
> >
> >
> >
> > --
> > Rod Simpson
> > @rockerston
> > rodsimpson.com <http://rodsimpson.com/>
> 

Re: [DISCUSS] Two dot o branch to Master

[Dave Johnson <sn...@gmail.com>]

As I said, I am +1 for merging two-dot-o to master and for doing all development against master with a JIRA issue, a pull request and a code-review for each new feature added.

There are some major changes in the two-dot-o branch that everybody should be aware of before we make this merge happen:

1. New Core Persistence System including these modules:

   - Common: common code and utilities for Core Persistence
   - Model: new Entity and Fields model
   - Collections: Astyanax and RxJava-based Entity storage and retrieval using MVCC algorithms
   - Query Index: Elastic Search based indexing and search of Entities 
   - Queue:  AWS SQS based distributing queueing system
   - Map: Astyanax-based Dictionary key and value storage

2. New CpEntityManager that uses Core Persistence instead of Hector. Unfortunately, there is still some Hector code in the new CpEntityManager that needs to be replaced with Core Persistence calls.

3. Changes to build process that require you to run local Cassandra and ElasticSearch when running tests (because we never could get tests to run consistently with embedded Cassandra).

4. The new CpEntityManager stores data in a format that is different from Usergrid 1.0. So any existing 1.0 system will have to be migrated before it can be used with two-dot-o. There is currently no migration system, however we have been working on a new import and export facility which should make it possible to move 1.0 data to two-dot-o. My understanding is that the import/export work is not complete.

The reason for the above changes is performance, two-dot-o gives us much better through-put than Usergrid 1.0. The two-dot-o work should also make Usergrid easier to maintain, but that benefit will not really kick-in until we remove the remaining Hector code from the CpEntityManager — the mix of Core Persistence and Hector code is not pleasant to work with.

- Dave








> On Thu, Nov 20, 2014 at 1:23 AM, Todd Nine <todd.nine@gmail.com <ma...@gmail.com>> wrote:
> I also agree that we should move our 2.0 branch into master, since it is
> now the primary focus of the project.
> 
> I'm unsure what the official policy surrounding releases and branches are.
> IMHO a branch in git is a branch, mater or otherwise, tags to preserve the
> commit we release from are what is truly important.
> 
> If we do 1 or 2 more maintenance releases of 1.0 and we're clear with our
> tagging of those releases, I feel a "one-dot-o" branch would be sufficient.
> 
> Lewis and Jake, any guidance here?
> 
> Thanks,
> Todd
> 
> On Wed Nov 19 2014 at 5:05:48 PM Rod Simpson <rod@rodsimpson.com <ma...@rodsimpson.com>> wrote:
> 
> > Usergriders,
> >
> > As most of you know, we have been working on upgrading the entity manager
> > and some other core changes.  This work has been taking place in the
> > two-dot-o branch.  The good news is that the 2.0 system is much, much
> > faster than 1.0.  Additionally, the use of ElasticSearch opens up a lot of
> > avenues for us in terms of adding new query grammar.
> >
> > We are now ready to move this work into master.  I am proposing that we
> > put the current master into a branch called one-dot-o, then move the
> > two-dot-o code into master.  Here are the pros and cons as I see them:
> >
> > Pros:
> > This will bring the existing work (and there is a lot of work happening)
> > to a more visible platform.  All changes will have to be brought in via PR
> > and it will allow us to scrutinize the code and work being done more
> > carefully.  It also validates the 2.0 codebase as the primary focus of the
> > project.
> >
> >
> > Cons:
> > We don’t currently have an upgrade path from 1.0 to 2.0.  This is
> > something that we intend to build but it may take some time to accomplish
> > (currently we are thinking it is about 2-3 months out, possibly sooner).
> > It also could make doing additional releases more difficult (e.g. can we
> > release from a branch that isn’t master?)  We are ready to release 1.0.1 as
> > we have had quite a few contributions, so if we have to release from
> > master, then this release needs to happen before putting the 2.0 code in.
> >
> >
> > Thoughts?
> >
> > Rod
> >
> >
> >
> > --
> > Rod Simpson
> > @rockerston
> > rodsimpson.com <http://rodsimpson.com/>
> 

Re: [DISCUSS] Two dot o branch to Master

[Todd Nine <to...@gmail.com>]

I also agree that we should move our 2.0 branch into master, since it is
now the primary focus of the project.

I'm unsure what the official policy surrounding releases and branches are.
IMHO a branch in git is a branch, mater or otherwise, tags to preserve the
commit we release from are what is truly important.

If we do 1 or 2 more maintenance releases of 1.0 and we're clear with our
tagging of those releases, I feel a "one-dot-o" branch would be sufficient.

Lewis and Jake, any guidance here?

Thanks,
Todd

On Wed Nov 19 2014 at 5:05:48 PM Rod Simpson <ro...@rodsimpson.com> wrote:

> Usergriders,
>
> As most of you know, we have been working on upgrading the entity manager
> and some other core changes.  This work has been taking place in the
> two-dot-o branch.  The good news is that the 2.0 system is much, much
> faster than 1.0.  Additionally, the use of ElasticSearch opens up a lot of
> avenues for us in terms of adding new query grammar.
>
> We are now ready to move this work into master.  I am proposing that we
> put the current master into a branch called one-dot-o, then move the
> two-dot-o code into master.  Here are the pros and cons as I see them:
>
> Pros:
> This will bring the existing work (and there is a lot of work happening)
> to a more visible platform.  All changes will have to be brought in via PR
> and it will allow us to scrutinize the code and work being done more
> carefully.  It also validates the 2.0 codebase as the primary focus of the
> project.
>
>
> Cons:
> We don’t currently have an upgrade path from 1.0 to 2.0.  This is
> something that we intend to build but it may take some time to accomplish
> (currently we are thinking it is about 2-3 months out, possibly sooner).
> It also could make doing additional releases more difficult (e.g. can we
> release from a branch that isn’t master?)  We are ready to release 1.0.1 as
> we have had quite a few contributions, so if we have to release from
> master, then this release needs to happen before putting the 2.0 code in.
>
>
> Thoughts?
>
> Rod
>
>
>
> --
> Rod Simpson
> @rockerston
> rodsimpson.com