Side Car New Repo vs not

[sankalp kohli <ko...@gmail.com>]

Hi,
    I am starting a new thread to get consensus on where the side car
should be contributed.

Please send your responses with pro/cons of each approach or any other
approach. Please be clear which approach you will pick while still giving
pros/cons of both approaches.

Thanks.
Sankalp

Re: Side Car New Repo vs not

[Jason Brown <ja...@gmail.com>]

+1 for separate repo. For pretty much all the same reasons Aleksey
elucidated.

On Tue, Aug 21, 2018 at 10:20 AM, Aleksey Yeshchenko <al...@apple.com>
wrote:

> Sure, allow me to elaborate - at least a little bit. But before I do, just
> let me note that this wasn’t a veto -1, just a shorthand for “I don’t like
> this option”.
>
> It would be nice to have sidecar and C* version and release cycles fully
> decoupled. I know it *can* be done when in-tree, but the way we vote on
> releases with tags off current branches would have to change somehow.
> Probably painfully. It would be nice to be able to easily enforce freezes,
> like the upcoming one, on the whole C* repo, while allowing feature
> development on the sidecar. It would be nice to not have sidecar commits in
> emails from commits@ mailing list. It would be nice to not have C* CI
> trigger necessarily on sidecar commits. Groups of people working on the two
> repos will mostly be different too, so what’s the point in sharing the repo?
>
> Having an extra repo with its own set of branches is cheap and easy - we
> already do that with dtests. I like cleanly separated things when coupling
> is avoidable. As such I would prefer the sidecar to live in a separate new
> repo, while still being part of the C* project.
>
> —
> AY
>
> On 21 August 2018 at 17:06:39, sankalp kohli (kohlisankalp@gmail.com)
> wrote:
>
> Hi Aleksey,
> Can you please elaborate on the reasons for your -1? This
> way we can make progress towards any one approach.
> Thanks,
> Sankalp
>
> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <al...@apple.com>
> wrote:
>
> > FWIW I’m strongly -1 on in-tree approach, and would much prefer a
> separate
> > repo, dtest-style.
> >
> > —
> > AY
> >
> > On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> > jeremiah.jordan@gmail.com) wrote:
> >
> > I think the following is a very big plus of it being in tree:
> > >> * Faster iteration speed in general. For example when we need to add
> a
> > >> new
> > >> JMX endpoint that the sidecar needs, or change something from JMX to
> a
> > >> virtual table (e.g. for repair, or monitoring) we can do all changes
> > >> including tests as one commit within the main repository and don't
> > have
> > >> to
> > >> commit to main repo, sidecar repo,
> >
> > I also don’t see a reason why the sidecar being in tree means it would
> not
> > work in a mixed version cluster. The nodes themselves must work in a
> mixed
> > version cluster during a rolling upgrade, I would expect any management
> > side car to operate in the same manor, in tree or not.
> >
> > This tool will be pretty tightly coupled with the server, and as
> someone
> > with experience developing such tightly coupled tools, it is *much*
> easier
> > to make sure you don’t accidentally break them if they are in tree. How
> > many times has someone updated some JMX interface, updated nodetool,
> and
> > then moved on? Breaking all the external tools not in tree, without
> > realizing it. The above point about being able to modify interfaces and
> the
> > side car in the same commit is huge in terms of making sure someone
> doesn’t
> > inadvertently break the side car while fixing something else.
> >
> > -Jeremiah
> >
> >
> > > On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>
> > wrote:
> > >
> > > Strongly agree with Blake. In my mind supporting multiple versions is
> > > mandatory. As I've stated before, we already do it with Reaper, I'd
> > > consider it a major misstep if we couldn't support multiple with the
> > > project - provided admin tool. It's the same reason dtests are
> separate
> > -
> > > they work with multiple versions.
> > >
> > > The number of repos does not affect distribution - if we want to ship
> > > Cassandra with the admin / repair tool (we should, imo), that can be
> > part
> > > of the build process.
> > >
> > >
> > >
> > >
> > > On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <be...@apple.com>
>
> > > wrote:
> > >
> > >> If the sidecar is going to be on a different release cadence, or
> > support
> > >> interacting with mixed mode clusters, then it should definitely be
> in
> > a
> > >> separate repo. I don’t even know how branching and merging would
> work
> > in a
> > >> repo that supports 2 separate release targets and/or mixed mode
> > >> compatibility, but I’m pretty sure it would be a mess.
> > >>
> > >> As a cluster management tool, mixed mode is probably going to be a
> goal
> > at
> > >> some point. As a new project, it will benefit from not being tied to
> > the C*
> > >> release cycle (which would probably delay any sidecar release until
> > >> whenever 4.1 is cut).
> > >>
> > >>
> > >> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.lynch@gmail.com)
>
> >
> > >> wrote:
> > >>
> > >> I think that the pros of incubating the sidecar in tree as a tool
> > first
> > >> outweigh the alternatives at this point of time. Rough tradeoffs
> that
> > I
> > >> see:
> > >>
> > >> Unique pros of in tree sidecar:
> > >> * Faster iteration speed in general. For example when we need to add
> a
> > >> new
> > >> JMX endpoint that the sidecar needs, or change something from JMX to
> a
> > >> virtual table (e.g. for repair, or monitoring) we can do all changes
> > >> including tests as one commit within the main repository and don't
> > have
> > >> to
> > >> commit to main repo, sidecar repo, and dtest repo (juggling version
> > >> compatibility along the way).
> > >> * We can in the future more easily move serious background
> > functionality
> > >> like compaction or repair itself (not repair scheduling, actual
> > >> repairing)
> > >> into the sidecar with a single atomic commit, we don't have to do
> two
> > >> phase
> > >> commits where we add some IPC mechanism to allow us to support it in
> > >> both,
> > >> then turn it on in the sidecar, then turn it off in the server,
> etc...
> > >> * I think that the verification is much easier (sounds like Jonathan
> > >> disagreed on the other thread, I could certainly be wrong), and we
> > don't
> > >> have to worry about testing matrices to assure that the sidecar
> works
> > >> with
> > >> various versions as the version of the sidecar that is released with
> > that
> > >> version of Cassandra is the only one we have to certify works. If
> > people
> > >> want to pull in new versions or maintain backports they can do that
> at
> > >> their discretion/testing.
> > >> * We can iterate and prove value before committing to a choice.
> Since
> > it
> > >> will be a separate artifact from the start we can always move the
> > >> artifact
> > >> to a separate repo later (but moving the other way is harder).
> > >> * Users will get the sidecar "for free" when they install the
> daemon,
> > >> they
> > >> don't need to take affirmative action to e.g. be able to restart
> their
> > >> cluster, run repair, or back their data up; it just comes out of the
> > box
> > >> for free.
> > >>
> > >> Unique pros of a separate repository sidecar:
> > >> * We can use a more modern build system like gradle instead of ant
> > >> * Merging changes is less "scary" I guess (I feel like if you're not
> > >> touching the daemon this is already true but I could see this being
> > less
> > >> worrisome for some).
> > >> * Releasing a separate artifact is somewhat easier from a separate
> > repo
> > >> (especially if we have gradle which makes e.g. building debs and
> rpms
> > >> trivial).
> > >> * We could backport to previous versions without getting into
> > arguments
> > >> about bug fixes vs features.
> > >> * Committers could be different from the main repo, which ... may be
> a
> > >> useful thing
> > >>
> > >> Non unique pros of a sidecar (could be achieved in the main repo or
> in
> > a
> > >> separate repo):
> > >> * A separate build artifact .jar/.deb/.rpm that can be installed
> > >> separately. It's slightly easier with a separate repo but certainly
> > not
> > >> out
> > >> of reach within a single repo (indeed the current patch already
> creates
> > a
> > >> separate jar, and we could create a separate .deb reasonably
> easily).
> > >> Personally I think having a separate .deb/.rpm is premature at this
> > point
> > >> (for companies that really want it they can build their own packages
> > >> using
> > >> the .jars), but I think it really is a distracting issue from where
> > the
> > >> patch should go as we can always choose to remove experimental .jar
> > files
> > >> that the main daemon doesn't touch.
> > >> * A separate process lifecycle. No matter where the sidecar goes, we
> > get
> > >> the benefit of restarting it being less dangerous for availability
> > than
> > >> restarting the main daemon.
> > >>
> > >> That all being said, these are strong opinions weakly held and I
> would
> > >> rather get something actually committed so that we can prove value
> one
> > >> way
> > >> or the other and am therefore, of course, happy to put sidecar
> patches
> > >> wherever someone can review and commit it.
> > >>
> > >> -Joey
> > >>
> > >> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <ko...@gmail.com>
>
> >
> > >> wrote:
> > >>
> > >>> Hi,
> > >>> I am starting a new thread to get consensus on where the side car
> > >>> should be contributed.
> > >>>
> > >>> Please send your responses with pro/cons of each approach or any
> > other
> > >>> approach. Please be clear which approach you will pick while still
> > >> giving
> > >>> pros/cons of both approaches.
> > >>>
> > >>> Thanks.
> > >>> Sankalp
> > >>>
> > >>
> > >
> > >
> > > --
> > > Jon Haddad
> > > http://www.rustyrazorblade.com
> > > twitter: rustyrazorblade
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
>

Re: Side Car New Repo vs not

[Sankalp Kohli <ko...@gmail.com>]

Thanks everyone for the feedback. Looks like we will go with separate repo as that is what majority of people prefer. 

Also note that we can always change this approach later as we build the side car. 

> On Aug 24, 2018, at 07:00, Eric Evans <jo...@gmail.com> wrote:
> 
>> On Thu, Aug 23, 2018 at 3:01 PM sankalp kohli <ko...@gmail.com> wrote:
>> 
>> Separate repo is in a majority so far. Please reply to this thread with
>> your responses.
> 
> I think it makes sense for the code, project, and workflows to be
> (de|loosely)-coupled, so the repo should be as well.
> 
> +1 for a separate repository
> 
>> On Tue, Aug 21, 2018 at 4:34 PM Rahul Singh <ra...@gmail.com>
>> wrote:
>> 
>>> +1 for separate repo. Especially on git. Maybe make it a submodule.
>>> 
>>> Rahul
>>> On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski <sp...@apache.org>,
>>> wrote:
>>>> I'm also currently -1 on the in-tree option.
>>>> 
>>>> Additionally to what Aleksey mentioned, I also don't see how we could
>>>> make this work with the current build and release process. Our scripts
>>>> [0] for creating releases (tarballs and native packages), would need
>>>> significant work to add support for an independent side-car. Our ant
>>>> based build process is also not a great start for adding new tasks, let
>>>> alone integrating other tool chains for web components for a potential
>>> UI.
>>>> 
>>>> [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git
>>>> 
>>>> 
>>>>> On 21.08.18 19:20, Aleksey Yeshchenko wrote:
>>>>> Sure, allow me to elaborate - at least a little bit. But before I do,
>>> just let me note that this wasn’t a veto -1, just a shorthand for “I don’t
>>> like this option”.
>>>>> 
>>>>> It would be nice to have sidecar and C* version and release cycles
>>> fully decoupled. I know it *can* be done when in-tree, but the way we vote
>>> on releases with tags off current branches would have to change somehow.
>>> Probably painfully. It would be nice to be able to easily enforce freezes,
>>> like the upcoming one, on the whole C* repo, while allowing feature
>>> development on the sidecar. It would be nice to not have sidecar commits in
>>> emails from commits@ mailing list. It would be nice to not have C* CI
>>> trigger necessarily on sidecar commits. Groups of people working on the two
>>> repos will mostly be different too, so what’s the point in sharing the repo?
>>>>> 
>>>>> Having an extra repo with its own set of branches is cheap and easy -
>>> we already do that with dtests. I like cleanly separated things when
>>> coupling is avoidable. As such I would prefer the sidecar to live in a
>>> separate new repo, while still being part of the C* project.
>>>>> 
>>>>> —
>>>>> AY
>>>>> 
>>>>> On 21 August 2018 at 17:06:39, sankalp kohli (kohlisankalp@gmail.com)
>>> wrote:
>>>>> 
>>>>> Hi Aleksey,
>>>>> Can you please elaborate on the reasons for your -1? This
>>>>> way we can make progress towards any one approach.
>>>>> Thanks,
>>>>> Sankalp
>>>>> 
>>>>> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <al...@apple.com>
>>>>> wrote:
>>>>> 
>>>>>> FWIW I’m strongly -1 on in-tree approach, and would much prefer a
>>> separate
>>>>>> repo, dtest-style.
>>>>>> 
>>>>>> —
>>>>>> AY
>>>>>> 
>>>>>> On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
>>>>>> jeremiah.jordan@gmail.com) wrote:
>>>>>> 
>>>>>> I think the following is a very big plus of it being in tree:
>>>>>>>> * Faster iteration speed in general. For example when we need to
>>> add a
>>>>>>>> new
>>>>>>>> JMX endpoint that the sidecar needs, or change something from
>>> JMX to a
>>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
>>> changes
>>>>>>>> including tests as one commit within the main repository and
>>> don't
>>>>>> have
>>>>>>>> to
>>>>>>>> commit to main repo, sidecar repo,
>>>>>> 
>>>>>> I also don’t see a reason why the sidecar being in tree means it
>>> would not
>>>>>> work in a mixed version cluster. The nodes themselves must work in a
>>> mixed
>>>>>> version cluster during a rolling upgrade, I would expect any
>>> management
>>>>>> side car to operate in the same manor, in tree or not.
>>>>>> 
>>>>>> This tool will be pretty tightly coupled with the server, and as
>>> someone
>>>>>> with experience developing such tightly coupled tools, it is *much*
>>> easier
>>>>>> to make sure you don’t accidentally break them if they are in tree.
>>> How
>>>>>> many times has someone updated some JMX interface, updated nodetool,
>>> and
>>>>>> then moved on? Breaking all the external tools not in tree, without
>>>>>> realizing it. The above point about being able to modify interfaces
>>> and the
>>>>>> side car in the same commit is huge in terms of making sure someone
>>> doesn’t
>>>>>> inadvertently break the side car while fixing something else.
>>>>>> 
>>>>>> -Jeremiah
>>>>>> 
>>>>>> 
>>>>>>> On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Strongly agree with Blake. In my mind supporting multiple versions
>>> is
>>>>>>> mandatory. As I've stated before, we already do it with Reaper, I'd
>>>>>>> consider it a major misstep if we couldn't support multiple with
>>> the
>>>>>>> project - provided admin tool. It's the same reason dtests are
>>> separate
>>>>>> -
>>>>>>> they work with multiple versions.
>>>>>>> 
>>>>>>> The number of repos does not affect distribution - if we want to
>>> ship
>>>>>>> Cassandra with the admin / repair tool (we should, imo), that can
>>> be
>>>>>> part
>>>>>>> of the build process.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <
>>> beggleston@apple.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> If the sidecar is going to be on a different release cadence, or
>>>>>> support
>>>>>>>> interacting with mixed mode clusters, then it should definitely
>>> be in
>>>>>> a
>>>>>>>> separate repo. I don’t even know how branching and merging would
>>> work
>>>>>> in a
>>>>>>>> repo that supports 2 separate release targets and/or mixed mode
>>>>>>>> compatibility, but I’m pretty sure it would be a mess.
>>>>>>>> 
>>>>>>>> As a cluster management tool, mixed mode is probably going to be
>>> a goal
>>>>>> at
>>>>>>>> some point. As a new project, it will benefit from not being
>>> tied to
>>>>>> the C*
>>>>>>>> release cycle (which would probably delay any sidecar release
>>> until
>>>>>>>> whenever 4.1 is cut).
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (
>>> joe.e.lynch@gmail.com)
>>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> I think that the pros of incubating the sidecar in tree as a tool
>>>>>> first
>>>>>>>> outweigh the alternatives at this point of time. Rough tradeoffs
>>> that
>>>>>> I
>>>>>>>> see:
>>>>>>>> 
>>>>>>>> Unique pros of in tree sidecar:
>>>>>>>> * Faster iteration speed in general. For example when we need to
>>> add a
>>>>>>>> new
>>>>>>>> JMX endpoint that the sidecar needs, or change something from
>>> JMX to a
>>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
>>> changes
>>>>>>>> including tests as one commit within the main repository and
>>> don't
>>>>>> have
>>>>>>>> to
>>>>>>>> commit to main repo, sidecar repo, and dtest repo (juggling
>>> version
>>>>>>>> compatibility along the way).
>>>>>>>> * We can in the future more easily move serious background
>>>>>> functionality
>>>>>>>> like compaction or repair itself (not repair scheduling, actual
>>>>>>>> repairing)
>>>>>>>> into the sidecar with a single atomic commit, we don't have to
>>> do two
>>>>>>>> phase
>>>>>>>> commits where we add some IPC mechanism to allow us to support
>>> it in
>>>>>>>> both,
>>>>>>>> then turn it on in the sidecar, then turn it off in the server,
>>> etc...
>>>>>>>> * I think that the verification is much easier (sounds like
>>> Jonathan
>>>>>>>> disagreed on the other thread, I could certainly be wrong), and
>>> we
>>>>>> don't
>>>>>>>> have to worry about testing matrices to assure that the sidecar
>>> works
>>>>>>>> with
>>>>>>>> various versions as the version of the sidecar that is released
>>> with
>>>>>> that
>>>>>>>> version of Cassandra is the only one we have to certify works. If
>>>>>> people
>>>>>>>> want to pull in new versions or maintain backports they can do
>>> that at
>>>>>>>> their discretion/testing.
>>>>>>>> * We can iterate and prove value before committing to a choice.
>>> Since
>>>>>> it
>>>>>>>> will be a separate artifact from the start we can always move the
>>>>>>>> artifact
>>>>>>>> to a separate repo later (but moving the other way is harder).
>>>>>>>> * Users will get the sidecar "for free" when they install the
>>> daemon,
>>>>>>>> they
>>>>>>>> don't need to take affirmative action to e.g. be able to restart
>>> their
>>>>>>>> cluster, run repair, or back their data up; it just comes out of
>>> the
>>>>>> box
>>>>>>>> for free.
>>>>>>>> 
>>>>>>>> Unique pros of a separate repository sidecar:
>>>>>>>> * We can use a more modern build system like gradle instead of
>>> ant
>>>>>>>> * Merging changes is less "scary" I guess (I feel like if you're
>>> not
>>>>>>>> touching the daemon this is already true but I could see this
>>> being
>>>>>> less
>>>>>>>> worrisome for some).
>>>>>>>> * Releasing a separate artifact is somewhat easier from a
>>> separate
>>>>>> repo
>>>>>>>> (especially if we have gradle which makes e.g. building debs and
>>> rpms
>>>>>>>> trivial).
>>>>>>>> * We could backport to previous versions without getting into
>>>>>> arguments
>>>>>>>> about bug fixes vs features.
>>>>>>>> * Committers could be different from the main repo, which ...
>>> may be a
>>>>>>>> useful thing
>>>>>>>> 
>>>>>>>> Non unique pros of a sidecar (could be achieved in the main repo
>>> or in
>>>>>> a
>>>>>>>> separate repo):
>>>>>>>> * A separate build artifact .jar/.deb/.rpm that can be installed
>>>>>>>> separately. It's slightly easier with a separate repo but
>>> certainly
>>>>>> not
>>>>>>>> out
>>>>>>>> of reach within a single repo (indeed the current patch already
>>> creates
>>>>>> a
>>>>>>>> separate jar, and we could create a separate .deb reasonably
>>> easily).
>>>>>>>> Personally I think having a separate .deb/.rpm is premature at
>>> this
>>>>>> point
>>>>>>>> (for companies that really want it they can build their own
>>> packages
>>>>>>>> using
>>>>>>>> the .jars), but I think it really is a distracting issue from
>>> where
>>>>>> the
>>>>>>>> patch should go as we can always choose to remove experimental
>>> .jar
>>>>>> files
>>>>>>>> that the main daemon doesn't touch.
>>>>>>>> * A separate process lifecycle. No matter where the sidecar
>>> goes, we
>>>>>> get
>>>>>>>> the benefit of restarting it being less dangerous for
>>> availability
>>>>>> than
>>>>>>>> restarting the main daemon.
>>>>>>>> 
>>>>>>>> That all being said, these are strong opinions weakly held and I
>>> would
>>>>>>>> rather get something actually committed so that we can prove
>>> value one
>>>>>>>> way
>>>>>>>> or the other and am therefore, of course, happy to put sidecar
>>> patches
>>>>>>>> wherever someone can review and commit it.
>>>>>>>> 
>>>>>>>> -Joey
>>>>>>>> 
>>>>>>>> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <
>>> kohlisankalp@gmail.com>
>>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> I am starting a new thread to get consensus on where the side
>>> car
>>>>>>>>> should be contributed.
>>>>>>>>> 
>>>>>>>>> Please send your responses with pro/cons of each approach or
>>> any
>>>>>> other
>>>>>>>>> approach. Please be clear which approach you will pick while
>>> still
>>>>>>>> giving
>>>>>>>>> pros/cons of both approaches.
>>>>>>>>> 
>>>>>>>>> Thanks.
>>>>>>>>> Sankalp
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Jon Haddad
>>>>>>> http://www.rustyrazorblade.com
>>>>>>> twitter: rustyrazorblade
>>>>>> 
>>>>>> 
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>> 
>>> 
> 
> 
> 
> -- 
> Eric Evans
> john.eric.evans@gmail.com
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org

Re: Side Car New Repo vs not

[Eric Evans <jo...@gmail.com>]

On Thu, Aug 23, 2018 at 3:01 PM sankalp kohli <ko...@gmail.com> wrote:
>
> Separate repo is in a majority so far. Please reply to this thread with
> your responses.

I think it makes sense for the code, project, and workflows to be
(de|loosely)-coupled, so the repo should be as well.

+1 for a separate repository

> On Tue, Aug 21, 2018 at 4:34 PM Rahul Singh <ra...@gmail.com>
> wrote:
>
> > +1 for separate repo. Especially on git. Maybe make it a submodule.
> >
> > Rahul
> > On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski <sp...@apache.org>,
> > wrote:
> > > I'm also currently -1 on the in-tree option.
> > >
> > > Additionally to what Aleksey mentioned, I also don't see how we could
> > > make this work with the current build and release process. Our scripts
> > > [0] for creating releases (tarballs and native packages), would need
> > > significant work to add support for an independent side-car. Our ant
> > > based build process is also not a great start for adding new tasks, let
> > > alone integrating other tool chains for web components for a potential
> > UI.
> > >
> > > [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git
> > >
> > >
> > > On 21.08.18 19:20, Aleksey Yeshchenko wrote:
> > > > Sure, allow me to elaborate - at least a little bit. But before I do,
> > just let me note that this wasn’t a veto -1, just a shorthand for “I don’t
> > like this option”.
> > > >
> > > > It would be nice to have sidecar and C* version and release cycles
> > fully decoupled. I know it *can* be done when in-tree, but the way we vote
> > on releases with tags off current branches would have to change somehow.
> > Probably painfully. It would be nice to be able to easily enforce freezes,
> > like the upcoming one, on the whole C* repo, while allowing feature
> > development on the sidecar. It would be nice to not have sidecar commits in
> > emails from commits@ mailing list. It would be nice to not have C* CI
> > trigger necessarily on sidecar commits. Groups of people working on the two
> > repos will mostly be different too, so what’s the point in sharing the repo?
> > > >
> > > > Having an extra repo with its own set of branches is cheap and easy -
> > we already do that with dtests. I like cleanly separated things when
> > coupling is avoidable. As such I would prefer the sidecar to live in a
> > separate new repo, while still being part of the C* project.
> > > >
> > > > —
> > > > AY
> > > >
> > > > On 21 August 2018 at 17:06:39, sankalp kohli (kohlisankalp@gmail.com)
> > wrote:
> > > >
> > > > Hi Aleksey,
> > > > Can you please elaborate on the reasons for your -1? This
> > > > way we can make progress towards any one approach.
> > > > Thanks,
> > > > Sankalp
> > > >
> > > > On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <al...@apple.com>
> > > > wrote:
> > > >
> > > > > FWIW I’m strongly -1 on in-tree approach, and would much prefer a
> > separate
> > > > > repo, dtest-style.
> > > > >
> > > > > —
> > > > > AY
> > > > >
> > > > > On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> > > > > jeremiah.jordan@gmail.com) wrote:
> > > > >
> > > > > I think the following is a very big plus of it being in tree:
> > > > > > > * Faster iteration speed in general. For example when we need to
> > add a
> > > > > > > new
> > > > > > > JMX endpoint that the sidecar needs, or change something from
> > JMX to a
> > > > > > > virtual table (e.g. for repair, or monitoring) we can do all
> > changes
> > > > > > > including tests as one commit within the main repository and
> > don't
> > > > > have
> > > > > > > to
> > > > > > > commit to main repo, sidecar repo,
> > > > >
> > > > > I also don’t see a reason why the sidecar being in tree means it
> > would not
> > > > > work in a mixed version cluster. The nodes themselves must work in a
> > mixed
> > > > > version cluster during a rolling upgrade, I would expect any
> > management
> > > > > side car to operate in the same manor, in tree or not.
> > > > >
> > > > > This tool will be pretty tightly coupled with the server, and as
> > someone
> > > > > with experience developing such tightly coupled tools, it is *much*
> > easier
> > > > > to make sure you don’t accidentally break them if they are in tree.
> > How
> > > > > many times has someone updated some JMX interface, updated nodetool,
> > and
> > > > > then moved on? Breaking all the external tools not in tree, without
> > > > > realizing it. The above point about being able to modify interfaces
> > and the
> > > > > side car in the same commit is huge in terms of making sure someone
> > doesn’t
> > > > > inadvertently break the side car while fixing something else.
> > > > >
> > > > > -Jeremiah
> > > > >
> > > > >
> > > > > > On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>
> > > > > wrote:
> > > > > >
> > > > > > Strongly agree with Blake. In my mind supporting multiple versions
> > is
> > > > > > mandatory. As I've stated before, we already do it with Reaper, I'd
> > > > > > consider it a major misstep if we couldn't support multiple with
> > the
> > > > > > project - provided admin tool. It's the same reason dtests are
> > separate
> > > > > -
> > > > > > they work with multiple versions.
> > > > > >
> > > > > > The number of repos does not affect distribution - if we want to
> > ship
> > > > > > Cassandra with the admin / repair tool (we should, imo), that can
> > be
> > > > > part
> > > > > > of the build process.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <
> > beggleston@apple.com>
> > > > > > wrote:
> > > > > >
> > > > > > > If the sidecar is going to be on a different release cadence, or
> > > > > support
> > > > > > > interacting with mixed mode clusters, then it should definitely
> > be in
> > > > > a
> > > > > > > separate repo. I don’t even know how branching and merging would
> > work
> > > > > in a
> > > > > > > repo that supports 2 separate release targets and/or mixed mode
> > > > > > > compatibility, but I’m pretty sure it would be a mess.
> > > > > > >
> > > > > > > As a cluster management tool, mixed mode is probably going to be
> > a goal
> > > > > at
> > > > > > > some point. As a new project, it will benefit from not being
> > tied to
> > > > > the C*
> > > > > > > release cycle (which would probably delay any sidecar release
> > until
> > > > > > > whenever 4.1 is cut).
> > > > > > >
> > > > > > >
> > > > > > > On August 20, 2018 at 3:22:54 PM, Joseph Lynch (
> > joe.e.lynch@gmail.com)
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > I think that the pros of incubating the sidecar in tree as a tool
> > > > > first
> > > > > > > outweigh the alternatives at this point of time. Rough tradeoffs
> > that
> > > > > I
> > > > > > > see:
> > > > > > >
> > > > > > > Unique pros of in tree sidecar:
> > > > > > > * Faster iteration speed in general. For example when we need to
> > add a
> > > > > > > new
> > > > > > > JMX endpoint that the sidecar needs, or change something from
> > JMX to a
> > > > > > > virtual table (e.g. for repair, or monitoring) we can do all
> > changes
> > > > > > > including tests as one commit within the main repository and
> > don't
> > > > > have
> > > > > > > to
> > > > > > > commit to main repo, sidecar repo, and dtest repo (juggling
> > version
> > > > > > > compatibility along the way).
> > > > > > > * We can in the future more easily move serious background
> > > > > functionality
> > > > > > > like compaction or repair itself (not repair scheduling, actual
> > > > > > > repairing)
> > > > > > > into the sidecar with a single atomic commit, we don't have to
> > do two
> > > > > > > phase
> > > > > > > commits where we add some IPC mechanism to allow us to support
> > it in
> > > > > > > both,
> > > > > > > then turn it on in the sidecar, then turn it off in the server,
> > etc...
> > > > > > > * I think that the verification is much easier (sounds like
> > Jonathan
> > > > > > > disagreed on the other thread, I could certainly be wrong), and
> > we
> > > > > don't
> > > > > > > have to worry about testing matrices to assure that the sidecar
> > works
> > > > > > > with
> > > > > > > various versions as the version of the sidecar that is released
> > with
> > > > > that
> > > > > > > version of Cassandra is the only one we have to certify works. If
> > > > > people
> > > > > > > want to pull in new versions or maintain backports they can do
> > that at
> > > > > > > their discretion/testing.
> > > > > > > * We can iterate and prove value before committing to a choice.
> > Since
> > > > > it
> > > > > > > will be a separate artifact from the start we can always move the
> > > > > > > artifact
> > > > > > > to a separate repo later (but moving the other way is harder).
> > > > > > > * Users will get the sidecar "for free" when they install the
> > daemon,
> > > > > > > they
> > > > > > > don't need to take affirmative action to e.g. be able to restart
> > their
> > > > > > > cluster, run repair, or back their data up; it just comes out of
> > the
> > > > > box
> > > > > > > for free.
> > > > > > >
> > > > > > > Unique pros of a separate repository sidecar:
> > > > > > > * We can use a more modern build system like gradle instead of
> > ant
> > > > > > > * Merging changes is less "scary" I guess (I feel like if you're
> > not
> > > > > > > touching the daemon this is already true but I could see this
> > being
> > > > > less
> > > > > > > worrisome for some).
> > > > > > > * Releasing a separate artifact is somewhat easier from a
> > separate
> > > > > repo
> > > > > > > (especially if we have gradle which makes e.g. building debs and
> > rpms
> > > > > > > trivial).
> > > > > > > * We could backport to previous versions without getting into
> > > > > arguments
> > > > > > > about bug fixes vs features.
> > > > > > > * Committers could be different from the main repo, which ...
> > may be a
> > > > > > > useful thing
> > > > > > >
> > > > > > > Non unique pros of a sidecar (could be achieved in the main repo
> > or in
> > > > > a
> > > > > > > separate repo):
> > > > > > > * A separate build artifact .jar/.deb/.rpm that can be installed
> > > > > > > separately. It's slightly easier with a separate repo but
> > certainly
> > > > > not
> > > > > > > out
> > > > > > > of reach within a single repo (indeed the current patch already
> > creates
> > > > > a
> > > > > > > separate jar, and we could create a separate .deb reasonably
> > easily).
> > > > > > > Personally I think having a separate .deb/.rpm is premature at
> > this
> > > > > point
> > > > > > > (for companies that really want it they can build their own
> > packages
> > > > > > > using
> > > > > > > the .jars), but I think it really is a distracting issue from
> > where
> > > > > the
> > > > > > > patch should go as we can always choose to remove experimental
> > .jar
> > > > > files
> > > > > > > that the main daemon doesn't touch.
> > > > > > > * A separate process lifecycle. No matter where the sidecar
> > goes, we
> > > > > get
> > > > > > > the benefit of restarting it being less dangerous for
> > availability
> > > > > than
> > > > > > > restarting the main daemon.
> > > > > > >
> > > > > > > That all being said, these are strong opinions weakly held and I
> > would
> > > > > > > rather get something actually committed so that we can prove
> > value one
> > > > > > > way
> > > > > > > or the other and am therefore, of course, happy to put sidecar
> > patches
> > > > > > > wherever someone can review and commit it.
> > > > > > >
> > > > > > > -Joey
> > > > > > >
> > > > > > > On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <
> > kohlisankalp@gmail.com>
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > > I am starting a new thread to get consensus on where the side
> > car
> > > > > > > > should be contributed.
> > > > > > > >
> > > > > > > > Please send your responses with pro/cons of each approach or
> > any
> > > > > other
> > > > > > > > approach. Please be clear which approach you will pick while
> > still
> > > > > > > giving
> > > > > > > > pros/cons of both approaches.
> > > > > > > >
> > > > > > > > Thanks.
> > > > > > > > Sankalp
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Jon Haddad
> > > > > > http://www.rustyrazorblade.com
> > > > > > twitter: rustyrazorblade
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > >
> > > > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > >
> >



-- 
Eric Evans
john.eric.evans@gmail.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org

Re: Side Car New Repo vs not

[kurt greaves <ku...@instaclustr.com>]

+1 separate repo. I think in-tree only works if you're *not* supporting
multiple versions and each sidecar release is directly tied to a
corresponding C* release. Considering this case is also completely
achievable in a separate repo anyway with minimal overhead we may as well
start that way and see where it goes.

On 24 August 2018 at 17:33, Sam Tunnicliffe <sa...@beobal.com> wrote:

> +1 for a separate repo
>
> On Fri, 24 Aug 2018 at 06:40, Michael Shuler <mi...@pbandjelly.org>
> wrote:
>
> > +1 for a separate repository.
> >
> > Michael
> >
> > On 08/23/2018 07:30 PM, Murukesh Mohanan wrote:
> > > FWIW, I think it's possible to merge in a separate repository into a
> > > subdirectory while keeping git history, but I don't know if the other
> way
> > > will be possible if commits span other parts of the repo as well\*
> (which
> > > will likely happen sooner or later). So a separate repo is a choice we
> > can
> > > backtrack from if it proves problematic later.
> > >
> > >
> > > \* it may be possible, but the commit messages might not make much
> sense
> > > after that.
> > >
> > > On Fri, 24 Aug 2018, 09:12 Benedict Elliott Smith, <
> benedict@apache.org>
> > > wrote:
> > >
> > >> +1 also for separate repo
> > >>
> > >>> On 24 Aug 2018, at 01:11, Jeff Jirsa <jj...@gmail.com> wrote:
> > >>>
> > >>> +1 for separate repo
> > >>>
> > >>>
> > >>> --
> > >>> Jeff Jirsa
> > >>>
> > >>>
> > >>>> On Aug 23, 2018, at 1:00 PM, sankalp kohli <ko...@gmail.com>
> > >> wrote:
> > >>>>
> > >>>> Separate repo is in a majority so far. Please reply to this thread
> > with
> > >>>> your responses.
> > >>>>
> > >>>> On Tue, Aug 21, 2018 at 4:34 PM Rahul Singh <
> > >> rahul.xavier.singh@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>>> +1 for separate repo. Especially on git. Maybe make it a submodule.
> > >>>>>
> > >>>>> Rahul
> > >>>>> On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski <
> spod@apache.org
> > >,
> > >>>>> wrote:
> > >>>>>> I'm also currently -1 on the in-tree option.
> > >>>>>>
> > >>>>>> Additionally to what Aleksey mentioned, I also don't see how we
> > could
> > >>>>>> make this work with the current build and release process. Our
> > scripts
> > >>>>>> [0] for creating releases (tarballs and native packages), would
> need
> > >>>>>> significant work to add support for an independent side-car. Our
> ant
> > >>>>>> based build process is also not a great start for adding new
> tasks,
> > >> let
> > >>>>>> alone integrating other tool chains for web components for a
> > potential
> > >>>>> UI.
> > >>>>>>
> > >>>>>> [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.
> git
> > >>>>>>
> > >>>>>>
> > >>>>>>> On 21.08.18 19:20, Aleksey Yeshchenko wrote:
> > >>>>>>> Sure, allow me to elaborate - at least a little bit. But before I
> > do,
> > >>>>> just let me note that this wasn’t a veto -1, just a shorthand for
> “I
> > >> don’t
> > >>>>> like this option”.
> > >>>>>>>
> > >>>>>>> It would be nice to have sidecar and C* version and release
> cycles
> > >>>>> fully decoupled. I know it *can* be done when in-tree, but the way
> we
> > >> vote
> > >>>>> on releases with tags off current branches would have to change
> > >> somehow.
> > >>>>> Probably painfully. It would be nice to be able to easily enforce
> > >> freezes,
> > >>>>> like the upcoming one, on the whole C* repo, while allowing feature
> > >>>>> development on the sidecar. It would be nice to not have sidecar
> > >> commits in
> > >>>>> emails from commits@ mailing list. It would be nice to not have C*
> > CI
> > >>>>> trigger necessarily on sidecar commits. Groups of people working on
> > >> the two
> > >>>>> repos will mostly be different too, so what’s the point in sharing
> > the
> > >> repo?
> > >>>>>>>
> > >>>>>>> Having an extra repo with its own set of branches is cheap and
> > easy -
> > >>>>> we already do that with dtests. I like cleanly separated things
> when
> > >>>>> coupling is avoidable. As such I would prefer the sidecar to live
> in
> > a
> > >>>>> separate new repo, while still being part of the C* project.
> > >>>>>>>
> > >>>>>>> —
> > >>>>>>> AY
> > >>>>>>>
> > >>>>>>> On 21 August 2018 at 17:06:39, sankalp kohli (
> > kohlisankalp@gmail.com
> > >> )
> > >>>>> wrote:
> > >>>>>>>
> > >>>>>>> Hi Aleksey,
> > >>>>>>> Can you please elaborate on the reasons for your -1? This
> > >>>>>>> way we can make progress towards any one approach.
> > >>>>>>> Thanks,
> > >>>>>>> Sankalp
> > >>>>>>>
> > >>>>>>> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <
> > >> aleksey@apple.com>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> FWIW I’m strongly -1 on in-tree approach, and would much prefer
> a
> > >>>>> separate
> > >>>>>>>> repo, dtest-style.
> > >>>>>>>>
> > >>>>>>>> —
> > >>>>>>>> AY
> > >>>>>>>>
> > >>>>>>>> On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> > >>>>>>>> jeremiah.jordan@gmail.com) wrote:
> > >>>>>>>>
> > >>>>>>>> I think the following is a very big plus of it being in tree:
> > >>>>>>>>>> * Faster iteration speed in general. For example when we need
> to
> > >>>>> add a
> > >>>>>>>>>> new
> > >>>>>>>>>> JMX endpoint that the sidecar needs, or change something from
> > >>>>> JMX to a
> > >>>>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
> > >>>>> changes
> > >>>>>>>>>> including tests as one commit within the main repository and
> > >>>>> don't
> > >>>>>>>> have
> > >>>>>>>>>> to
> > >>>>>>>>>> commit to main repo, sidecar repo,
> > >>>>>>>>
> > >>>>>>>> I also don’t see a reason why the sidecar being in tree means it
> > >>>>> would not
> > >>>>>>>> work in a mixed version cluster. The nodes themselves must work
> > in a
> > >>>>> mixed
> > >>>>>>>> version cluster during a rolling upgrade, I would expect any
> > >>>>> management
> > >>>>>>>> side car to operate in the same manor, in tree or not.
> > >>>>>>>>
> > >>>>>>>> This tool will be pretty tightly coupled with the server, and as
> > >>>>> someone
> > >>>>>>>> with experience developing such tightly coupled tools, it is
> > *much*
> > >>>>> easier
> > >>>>>>>> to make sure you don’t accidentally break them if they are in
> > tree.
> > >>>>> How
> > >>>>>>>> many times has someone updated some JMX interface, updated
> > nodetool,
> > >>>>> and
> > >>>>>>>> then moved on? Breaking all the external tools not in tree,
> > without
> > >>>>>>>> realizing it. The above point about being able to modify
> > interfaces
> > >>>>> and the
> > >>>>>>>> side car in the same commit is huge in terms of making sure
> > someone
> > >>>>> doesn’t
> > >>>>>>>> inadvertently break the side car while fixing something else.
> > >>>>>>>>
> > >>>>>>>> -Jeremiah
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <
> jon@jonhaddad.com
> > >
> > >>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Strongly agree with Blake. In my mind supporting multiple
> > versions
> > >>>>> is
> > >>>>>>>>> mandatory. As I've stated before, we already do it with Reaper,
> > I'd
> > >>>>>>>>> consider it a major misstep if we couldn't support multiple
> with
> > >>>>> the
> > >>>>>>>>> project - provided admin tool. It's the same reason dtests are
> > >>>>> separate
> > >>>>>>>> -
> > >>>>>>>>> they work with multiple versions.
> > >>>>>>>>>
> > >>>>>>>>> The number of repos does not affect distribution - if we want
> to
> > >>>>> ship
> > >>>>>>>>> Cassandra with the admin / repair tool (we should, imo), that
> can
> > >>>>> be
> > >>>>>>>> part
> > >>>>>>>>> of the build process.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <
> > >>>>> beggleston@apple.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> If the sidecar is going to be on a different release cadence,
> or
> > >>>>>>>> support
> > >>>>>>>>>> interacting with mixed mode clusters, then it should
> definitely
> > >>>>> be in
> > >>>>>>>> a
> > >>>>>>>>>> separate repo. I don’t even know how branching and merging
> would
> > >>>>> work
> > >>>>>>>> in a
> > >>>>>>>>>> repo that supports 2 separate release targets and/or mixed
> mode
> > >>>>>>>>>> compatibility, but I’m pretty sure it would be a mess.
> > >>>>>>>>>>
> > >>>>>>>>>> As a cluster management tool, mixed mode is probably going to
> be
> > >>>>> a goal
> > >>>>>>>> at
> > >>>>>>>>>> some point. As a new project, it will benefit from not being
> > >>>>> tied to
> > >>>>>>>> the C*
> > >>>>>>>>>> release cycle (which would probably delay any sidecar release
> > >>>>> until
> > >>>>>>>>>> whenever 4.1 is cut).
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (
> > >>>>> joe.e.lynch@gmail.com)
> > >>>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> I think that the pros of incubating the sidecar in tree as a
> > tool
> > >>>>>>>> first
> > >>>>>>>>>> outweigh the alternatives at this point of time. Rough
> tradeoffs
> > >>>>> that
> > >>>>>>>> I
> > >>>>>>>>>> see:
> > >>>>>>>>>>
> > >>>>>>>>>> Unique pros of in tree sidecar:
> > >>>>>>>>>> * Faster iteration speed in general. For example when we need
> to
> > >>>>> add a
> > >>>>>>>>>> new
> > >>>>>>>>>> JMX endpoint that the sidecar needs, or change something from
> > >>>>> JMX to a
> > >>>>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
> > >>>>> changes
> > >>>>>>>>>> including tests as one commit within the main repository and
> > >>>>> don't
> > >>>>>>>> have
> > >>>>>>>>>> to
> > >>>>>>>>>> commit to main repo, sidecar repo, and dtest repo (juggling
> > >>>>> version
> > >>>>>>>>>> compatibility along the way).
> > >>>>>>>>>> * We can in the future more easily move serious background
> > >>>>>>>> functionality
> > >>>>>>>>>> like compaction or repair itself (not repair scheduling,
> actual
> > >>>>>>>>>> repairing)
> > >>>>>>>>>> into the sidecar with a single atomic commit, we don't have to
> > >>>>> do two
> > >>>>>>>>>> phase
> > >>>>>>>>>> commits where we add some IPC mechanism to allow us to support
> > >>>>> it in
> > >>>>>>>>>> both,
> > >>>>>>>>>> then turn it on in the sidecar, then turn it off in the
> server,
> > >>>>> etc...
> > >>>>>>>>>> * I think that the verification is much easier (sounds like
> > >>>>> Jonathan
> > >>>>>>>>>> disagreed on the other thread, I could certainly be wrong),
> and
> > >>>>> we
> > >>>>>>>> don't
> > >>>>>>>>>> have to worry about testing matrices to assure that the
> sidecar
> > >>>>> works
> > >>>>>>>>>> with
> > >>>>>>>>>> various versions as the version of the sidecar that is
> released
> > >>>>> with
> > >>>>>>>> that
> > >>>>>>>>>> version of Cassandra is the only one we have to certify works.
> > If
> > >>>>>>>> people
> > >>>>>>>>>> want to pull in new versions or maintain backports they can do
> > >>>>> that at
> > >>>>>>>>>> their discretion/testing.
> > >>>>>>>>>> * We can iterate and prove value before committing to a
> choice.
> > >>>>> Since
> > >>>>>>>> it
> > >>>>>>>>>> will be a separate artifact from the start we can always move
> > the
> > >>>>>>>>>> artifact
> > >>>>>>>>>> to a separate repo later (but moving the other way is harder).
> > >>>>>>>>>> * Users will get the sidecar "for free" when they install the
> > >>>>> daemon,
> > >>>>>>>>>> they
> > >>>>>>>>>> don't need to take affirmative action to e.g. be able to
> restart
> > >>>>> their
> > >>>>>>>>>> cluster, run repair, or back their data up; it just comes out
> of
> > >>>>> the
> > >>>>>>>> box
> > >>>>>>>>>> for free.
> > >>>>>>>>>>
> > >>>>>>>>>> Unique pros of a separate repository sidecar:
> > >>>>>>>>>> * We can use a more modern build system like gradle instead of
> > >>>>> ant
> > >>>>>>>>>> * Merging changes is less "scary" I guess (I feel like if
> you're
> > >>>>> not
> > >>>>>>>>>> touching the daemon this is already true but I could see this
> > >>>>> being
> > >>>>>>>> less
> > >>>>>>>>>> worrisome for some).
> > >>>>>>>>>> * Releasing a separate artifact is somewhat easier from a
> > >>>>> separate
> > >>>>>>>> repo
> > >>>>>>>>>> (especially if we have gradle which makes e.g. building debs
> and
> > >>>>> rpms
> > >>>>>>>>>> trivial).
> > >>>>>>>>>> * We could backport to previous versions without getting into
> > >>>>>>>> arguments
> > >>>>>>>>>> about bug fixes vs features.
> > >>>>>>>>>> * Committers could be different from the main repo, which ...
> > >>>>> may be a
> > >>>>>>>>>> useful thing
> > >>>>>>>>>>
> > >>>>>>>>>> Non unique pros of a sidecar (could be achieved in the main
> repo
> > >>>>> or in
> > >>>>>>>> a
> > >>>>>>>>>> separate repo):
> > >>>>>>>>>> * A separate build artifact .jar/.deb/.rpm that can be
> installed
> > >>>>>>>>>> separately. It's slightly easier with a separate repo but
> > >>>>> certainly
> > >>>>>>>> not
> > >>>>>>>>>> out
> > >>>>>>>>>> of reach within a single repo (indeed the current patch
> already
> > >>>>> creates
> > >>>>>>>> a
> > >>>>>>>>>> separate jar, and we could create a separate .deb reasonably
> > >>>>> easily).
> > >>>>>>>>>> Personally I think having a separate .deb/.rpm is premature at
> > >>>>> this
> > >>>>>>>> point
> > >>>>>>>>>> (for companies that really want it they can build their own
> > >>>>> packages
> > >>>>>>>>>> using
> > >>>>>>>>>> the .jars), but I think it really is a distracting issue from
> > >>>>> where
> > >>>>>>>> the
> > >>>>>>>>>> patch should go as we can always choose to remove experimental
> > >>>>> .jar
> > >>>>>>>> files
> > >>>>>>>>>> that the main daemon doesn't touch.
> > >>>>>>>>>> * A separate process lifecycle. No matter where the sidecar
> > >>>>> goes, we
> > >>>>>>>> get
> > >>>>>>>>>> the benefit of restarting it being less dangerous for
> > >>>>> availability
> > >>>>>>>> than
> > >>>>>>>>>> restarting the main daemon.
> > >>>>>>>>>>
> > >>>>>>>>>> That all being said, these are strong opinions weakly held
> and I
> > >>>>> would
> > >>>>>>>>>> rather get something actually committed so that we can prove
> > >>>>> value one
> > >>>>>>>>>> way
> > >>>>>>>>>> or the other and am therefore, of course, happy to put sidecar
> > >>>>> patches
> > >>>>>>>>>> wherever someone can review and commit it.
> > >>>>>>>>>>
> > >>>>>>>>>> -Joey
> > >>>>>>>>>>
> > >>>>>>>>>> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <
> > >>>>> kohlisankalp@gmail.com>
> > >>>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Hi,
> > >>>>>>>>>>> I am starting a new thread to get consensus on where the side
> > >>>>> car
> > >>>>>>>>>>> should be contributed.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Please send your responses with pro/cons of each approach or
> > >>>>> any
> > >>>>>>>> other
> > >>>>>>>>>>> approach. Please be clear which approach you will pick while
> > >>>>> still
> > >>>>>>>>>> giving
> > >>>>>>>>>>> pros/cons of both approaches.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks.
> > >>>>>>>>>>> Sankalp
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> Jon Haddad
> > >>>>>>>>> http://www.rustyrazorblade.com
> > >>>>>>>>> twitter: rustyrazorblade
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >> ---------------------------------------------------------------------
> > >>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > ---------------------------------------------------------------------
> > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>>>>>
> > >>>>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>>
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > >> For additional commands, e-mail: dev-help@cassandra.apache.org
> > >>
> > >> --
> > >
> > > Muru
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
>

Re: Side Car New Repo vs not

[Sam Tunnicliffe <sa...@beobal.com>]

+1 for a separate repo

On Fri, 24 Aug 2018 at 06:40, Michael Shuler <mi...@pbandjelly.org> wrote:

> +1 for a separate repository.
>
> Michael
>
> On 08/23/2018 07:30 PM, Murukesh Mohanan wrote:
> > FWIW, I think it's possible to merge in a separate repository into a
> > subdirectory while keeping git history, but I don't know if the other way
> > will be possible if commits span other parts of the repo as well\* (which
> > will likely happen sooner or later). So a separate repo is a choice we
> can
> > backtrack from if it proves problematic later.
> >
> >
> > \* it may be possible, but the commit messages might not make much sense
> > after that.
> >
> > On Fri, 24 Aug 2018, 09:12 Benedict Elliott Smith, <be...@apache.org>
> > wrote:
> >
> >> +1 also for separate repo
> >>
> >>> On 24 Aug 2018, at 01:11, Jeff Jirsa <jj...@gmail.com> wrote:
> >>>
> >>> +1 for separate repo
> >>>
> >>>
> >>> --
> >>> Jeff Jirsa
> >>>
> >>>
> >>>> On Aug 23, 2018, at 1:00 PM, sankalp kohli <ko...@gmail.com>
> >> wrote:
> >>>>
> >>>> Separate repo is in a majority so far. Please reply to this thread
> with
> >>>> your responses.
> >>>>
> >>>> On Tue, Aug 21, 2018 at 4:34 PM Rahul Singh <
> >> rahul.xavier.singh@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> +1 for separate repo. Especially on git. Maybe make it a submodule.
> >>>>>
> >>>>> Rahul
> >>>>> On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski <spod@apache.org
> >,
> >>>>> wrote:
> >>>>>> I'm also currently -1 on the in-tree option.
> >>>>>>
> >>>>>> Additionally to what Aleksey mentioned, I also don't see how we
> could
> >>>>>> make this work with the current build and release process. Our
> scripts
> >>>>>> [0] for creating releases (tarballs and native packages), would need
> >>>>>> significant work to add support for an independent side-car. Our ant
> >>>>>> based build process is also not a great start for adding new tasks,
> >> let
> >>>>>> alone integrating other tool chains for web components for a
> potential
> >>>>> UI.
> >>>>>>
> >>>>>> [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git
> >>>>>>
> >>>>>>
> >>>>>>> On 21.08.18 19:20, Aleksey Yeshchenko wrote:
> >>>>>>> Sure, allow me to elaborate - at least a little bit. But before I
> do,
> >>>>> just let me note that this wasn’t a veto -1, just a shorthand for “I
> >> don’t
> >>>>> like this option”.
> >>>>>>>
> >>>>>>> It would be nice to have sidecar and C* version and release cycles
> >>>>> fully decoupled. I know it *can* be done when in-tree, but the way we
> >> vote
> >>>>> on releases with tags off current branches would have to change
> >> somehow.
> >>>>> Probably painfully. It would be nice to be able to easily enforce
> >> freezes,
> >>>>> like the upcoming one, on the whole C* repo, while allowing feature
> >>>>> development on the sidecar. It would be nice to not have sidecar
> >> commits in
> >>>>> emails from commits@ mailing list. It would be nice to not have C*
> CI
> >>>>> trigger necessarily on sidecar commits. Groups of people working on
> >> the two
> >>>>> repos will mostly be different too, so what’s the point in sharing
> the
> >> repo?
> >>>>>>>
> >>>>>>> Having an extra repo with its own set of branches is cheap and
> easy -
> >>>>> we already do that with dtests. I like cleanly separated things when
> >>>>> coupling is avoidable. As such I would prefer the sidecar to live in
> a
> >>>>> separate new repo, while still being part of the C* project.
> >>>>>>>
> >>>>>>> —
> >>>>>>> AY
> >>>>>>>
> >>>>>>> On 21 August 2018 at 17:06:39, sankalp kohli (
> kohlisankalp@gmail.com
> >> )
> >>>>> wrote:
> >>>>>>>
> >>>>>>> Hi Aleksey,
> >>>>>>> Can you please elaborate on the reasons for your -1? This
> >>>>>>> way we can make progress towards any one approach.
> >>>>>>> Thanks,
> >>>>>>> Sankalp
> >>>>>>>
> >>>>>>> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <
> >> aleksey@apple.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> FWIW I’m strongly -1 on in-tree approach, and would much prefer a
> >>>>> separate
> >>>>>>>> repo, dtest-style.
> >>>>>>>>
> >>>>>>>> —
> >>>>>>>> AY
> >>>>>>>>
> >>>>>>>> On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> >>>>>>>> jeremiah.jordan@gmail.com) wrote:
> >>>>>>>>
> >>>>>>>> I think the following is a very big plus of it being in tree:
> >>>>>>>>>> * Faster iteration speed in general. For example when we need to
> >>>>> add a
> >>>>>>>>>> new
> >>>>>>>>>> JMX endpoint that the sidecar needs, or change something from
> >>>>> JMX to a
> >>>>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
> >>>>> changes
> >>>>>>>>>> including tests as one commit within the main repository and
> >>>>> don't
> >>>>>>>> have
> >>>>>>>>>> to
> >>>>>>>>>> commit to main repo, sidecar repo,
> >>>>>>>>
> >>>>>>>> I also don’t see a reason why the sidecar being in tree means it
> >>>>> would not
> >>>>>>>> work in a mixed version cluster. The nodes themselves must work
> in a
> >>>>> mixed
> >>>>>>>> version cluster during a rolling upgrade, I would expect any
> >>>>> management
> >>>>>>>> side car to operate in the same manor, in tree or not.
> >>>>>>>>
> >>>>>>>> This tool will be pretty tightly coupled with the server, and as
> >>>>> someone
> >>>>>>>> with experience developing such tightly coupled tools, it is
> *much*
> >>>>> easier
> >>>>>>>> to make sure you don’t accidentally break them if they are in
> tree.
> >>>>> How
> >>>>>>>> many times has someone updated some JMX interface, updated
> nodetool,
> >>>>> and
> >>>>>>>> then moved on? Breaking all the external tools not in tree,
> without
> >>>>>>>> realizing it. The above point about being able to modify
> interfaces
> >>>>> and the
> >>>>>>>> side car in the same commit is huge in terms of making sure
> someone
> >>>>> doesn’t
> >>>>>>>> inadvertently break the side car while fixing something else.
> >>>>>>>>
> >>>>>>>> -Jeremiah
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jon@jonhaddad.com
> >
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Strongly agree with Blake. In my mind supporting multiple
> versions
> >>>>> is
> >>>>>>>>> mandatory. As I've stated before, we already do it with Reaper,
> I'd
> >>>>>>>>> consider it a major misstep if we couldn't support multiple with
> >>>>> the
> >>>>>>>>> project - provided admin tool. It's the same reason dtests are
> >>>>> separate
> >>>>>>>> -
> >>>>>>>>> they work with multiple versions.
> >>>>>>>>>
> >>>>>>>>> The number of repos does not affect distribution - if we want to
> >>>>> ship
> >>>>>>>>> Cassandra with the admin / repair tool (we should, imo), that can
> >>>>> be
> >>>>>>>> part
> >>>>>>>>> of the build process.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <
> >>>>> beggleston@apple.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> If the sidecar is going to be on a different release cadence, or
> >>>>>>>> support
> >>>>>>>>>> interacting with mixed mode clusters, then it should definitely
> >>>>> be in
> >>>>>>>> a
> >>>>>>>>>> separate repo. I don’t even know how branching and merging would
> >>>>> work
> >>>>>>>> in a
> >>>>>>>>>> repo that supports 2 separate release targets and/or mixed mode
> >>>>>>>>>> compatibility, but I’m pretty sure it would be a mess.
> >>>>>>>>>>
> >>>>>>>>>> As a cluster management tool, mixed mode is probably going to be
> >>>>> a goal
> >>>>>>>> at
> >>>>>>>>>> some point. As a new project, it will benefit from not being
> >>>>> tied to
> >>>>>>>> the C*
> >>>>>>>>>> release cycle (which would probably delay any sidecar release
> >>>>> until
> >>>>>>>>>> whenever 4.1 is cut).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (
> >>>>> joe.e.lynch@gmail.com)
> >>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I think that the pros of incubating the sidecar in tree as a
> tool
> >>>>>>>> first
> >>>>>>>>>> outweigh the alternatives at this point of time. Rough tradeoffs
> >>>>> that
> >>>>>>>> I
> >>>>>>>>>> see:
> >>>>>>>>>>
> >>>>>>>>>> Unique pros of in tree sidecar:
> >>>>>>>>>> * Faster iteration speed in general. For example when we need to
> >>>>> add a
> >>>>>>>>>> new
> >>>>>>>>>> JMX endpoint that the sidecar needs, or change something from
> >>>>> JMX to a
> >>>>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
> >>>>> changes
> >>>>>>>>>> including tests as one commit within the main repository and
> >>>>> don't
> >>>>>>>> have
> >>>>>>>>>> to
> >>>>>>>>>> commit to main repo, sidecar repo, and dtest repo (juggling
> >>>>> version
> >>>>>>>>>> compatibility along the way).
> >>>>>>>>>> * We can in the future more easily move serious background
> >>>>>>>> functionality
> >>>>>>>>>> like compaction or repair itself (not repair scheduling, actual
> >>>>>>>>>> repairing)
> >>>>>>>>>> into the sidecar with a single atomic commit, we don't have to
> >>>>> do two
> >>>>>>>>>> phase
> >>>>>>>>>> commits where we add some IPC mechanism to allow us to support
> >>>>> it in
> >>>>>>>>>> both,
> >>>>>>>>>> then turn it on in the sidecar, then turn it off in the server,
> >>>>> etc...
> >>>>>>>>>> * I think that the verification is much easier (sounds like
> >>>>> Jonathan
> >>>>>>>>>> disagreed on the other thread, I could certainly be wrong), and
> >>>>> we
> >>>>>>>> don't
> >>>>>>>>>> have to worry about testing matrices to assure that the sidecar
> >>>>> works
> >>>>>>>>>> with
> >>>>>>>>>> various versions as the version of the sidecar that is released
> >>>>> with
> >>>>>>>> that
> >>>>>>>>>> version of Cassandra is the only one we have to certify works.
> If
> >>>>>>>> people
> >>>>>>>>>> want to pull in new versions or maintain backports they can do
> >>>>> that at
> >>>>>>>>>> their discretion/testing.
> >>>>>>>>>> * We can iterate and prove value before committing to a choice.
> >>>>> Since
> >>>>>>>> it
> >>>>>>>>>> will be a separate artifact from the start we can always move
> the
> >>>>>>>>>> artifact
> >>>>>>>>>> to a separate repo later (but moving the other way is harder).
> >>>>>>>>>> * Users will get the sidecar "for free" when they install the
> >>>>> daemon,
> >>>>>>>>>> they
> >>>>>>>>>> don't need to take affirmative action to e.g. be able to restart
> >>>>> their
> >>>>>>>>>> cluster, run repair, or back their data up; it just comes out of
> >>>>> the
> >>>>>>>> box
> >>>>>>>>>> for free.
> >>>>>>>>>>
> >>>>>>>>>> Unique pros of a separate repository sidecar:
> >>>>>>>>>> * We can use a more modern build system like gradle instead of
> >>>>> ant
> >>>>>>>>>> * Merging changes is less "scary" I guess (I feel like if you're
> >>>>> not
> >>>>>>>>>> touching the daemon this is already true but I could see this
> >>>>> being
> >>>>>>>> less
> >>>>>>>>>> worrisome for some).
> >>>>>>>>>> * Releasing a separate artifact is somewhat easier from a
> >>>>> separate
> >>>>>>>> repo
> >>>>>>>>>> (especially if we have gradle which makes e.g. building debs and
> >>>>> rpms
> >>>>>>>>>> trivial).
> >>>>>>>>>> * We could backport to previous versions without getting into
> >>>>>>>> arguments
> >>>>>>>>>> about bug fixes vs features.
> >>>>>>>>>> * Committers could be different from the main repo, which ...
> >>>>> may be a
> >>>>>>>>>> useful thing
> >>>>>>>>>>
> >>>>>>>>>> Non unique pros of a sidecar (could be achieved in the main repo
> >>>>> or in
> >>>>>>>> a
> >>>>>>>>>> separate repo):
> >>>>>>>>>> * A separate build artifact .jar/.deb/.rpm that can be installed
> >>>>>>>>>> separately. It's slightly easier with a separate repo but
> >>>>> certainly
> >>>>>>>> not
> >>>>>>>>>> out
> >>>>>>>>>> of reach within a single repo (indeed the current patch already
> >>>>> creates
> >>>>>>>> a
> >>>>>>>>>> separate jar, and we could create a separate .deb reasonably
> >>>>> easily).
> >>>>>>>>>> Personally I think having a separate .deb/.rpm is premature at
> >>>>> this
> >>>>>>>> point
> >>>>>>>>>> (for companies that really want it they can build their own
> >>>>> packages
> >>>>>>>>>> using
> >>>>>>>>>> the .jars), but I think it really is a distracting issue from
> >>>>> where
> >>>>>>>> the
> >>>>>>>>>> patch should go as we can always choose to remove experimental
> >>>>> .jar
> >>>>>>>> files
> >>>>>>>>>> that the main daemon doesn't touch.
> >>>>>>>>>> * A separate process lifecycle. No matter where the sidecar
> >>>>> goes, we
> >>>>>>>> get
> >>>>>>>>>> the benefit of restarting it being less dangerous for
> >>>>> availability
> >>>>>>>> than
> >>>>>>>>>> restarting the main daemon.
> >>>>>>>>>>
> >>>>>>>>>> That all being said, these are strong opinions weakly held and I
> >>>>> would
> >>>>>>>>>> rather get something actually committed so that we can prove
> >>>>> value one
> >>>>>>>>>> way
> >>>>>>>>>> or the other and am therefore, of course, happy to put sidecar
> >>>>> patches
> >>>>>>>>>> wherever someone can review and commit it.
> >>>>>>>>>>
> >>>>>>>>>> -Joey
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <
> >>>>> kohlisankalp@gmail.com>
> >>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>> I am starting a new thread to get consensus on where the side
> >>>>> car
> >>>>>>>>>>> should be contributed.
> >>>>>>>>>>>
> >>>>>>>>>>> Please send your responses with pro/cons of each approach or
> >>>>> any
> >>>>>>>> other
> >>>>>>>>>>> approach. Please be clear which approach you will pick while
> >>>>> still
> >>>>>>>>>> giving
> >>>>>>>>>>> pros/cons of both approaches.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks.
> >>>>>>>>>>> Sankalp
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Jon Haddad
> >>>>>>>>> http://www.rustyrazorblade.com
> >>>>>>>>> twitter: rustyrazorblade
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >> ---------------------------------------------------------------------
> >>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>>
> >>>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>
> >> --
> >
> > Muru
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>

Re: Side Car New Repo vs not

[Michael Shuler <mi...@pbandjelly.org>]

+1 for a separate repository.

Michael

On 08/23/2018 07:30 PM, Murukesh Mohanan wrote:
> FWIW, I think it's possible to merge in a separate repository into a
> subdirectory while keeping git history, but I don't know if the other way
> will be possible if commits span other parts of the repo as well\* (which
> will likely happen sooner or later). So a separate repo is a choice we can
> backtrack from if it proves problematic later.
> 
> 
> \* it may be possible, but the commit messages might not make much sense
> after that.
> 
> On Fri, 24 Aug 2018, 09:12 Benedict Elliott Smith, <be...@apache.org>
> wrote:
> 
>> +1 also for separate repo
>>
>>> On 24 Aug 2018, at 01:11, Jeff Jirsa <jj...@gmail.com> wrote:
>>>
>>> +1 for separate repo
>>>
>>>
>>> --
>>> Jeff Jirsa
>>>
>>>
>>>> On Aug 23, 2018, at 1:00 PM, sankalp kohli <ko...@gmail.com>
>> wrote:
>>>>
>>>> Separate repo is in a majority so far. Please reply to this thread with
>>>> your responses.
>>>>
>>>> On Tue, Aug 21, 2018 at 4:34 PM Rahul Singh <
>> rahul.xavier.singh@gmail.com>
>>>> wrote:
>>>>
>>>>> +1 for separate repo. Especially on git. Maybe make it a submodule.
>>>>>
>>>>> Rahul
>>>>> On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski <sp...@apache.org>,
>>>>> wrote:
>>>>>> I'm also currently -1 on the in-tree option.
>>>>>>
>>>>>> Additionally to what Aleksey mentioned, I also don't see how we could
>>>>>> make this work with the current build and release process. Our scripts
>>>>>> [0] for creating releases (tarballs and native packages), would need
>>>>>> significant work to add support for an independent side-car. Our ant
>>>>>> based build process is also not a great start for adding new tasks,
>> let
>>>>>> alone integrating other tool chains for web components for a potential
>>>>> UI.
>>>>>>
>>>>>> [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git
>>>>>>
>>>>>>
>>>>>>> On 21.08.18 19:20, Aleksey Yeshchenko wrote:
>>>>>>> Sure, allow me to elaborate - at least a little bit. But before I do,
>>>>> just let me note that this wasn’t a veto -1, just a shorthand for “I
>> don’t
>>>>> like this option”.
>>>>>>>
>>>>>>> It would be nice to have sidecar and C* version and release cycles
>>>>> fully decoupled. I know it *can* be done when in-tree, but the way we
>> vote
>>>>> on releases with tags off current branches would have to change
>> somehow.
>>>>> Probably painfully. It would be nice to be able to easily enforce
>> freezes,
>>>>> like the upcoming one, on the whole C* repo, while allowing feature
>>>>> development on the sidecar. It would be nice to not have sidecar
>> commits in
>>>>> emails from commits@ mailing list. It would be nice to not have C* CI
>>>>> trigger necessarily on sidecar commits. Groups of people working on
>> the two
>>>>> repos will mostly be different too, so what’s the point in sharing the
>> repo?
>>>>>>>
>>>>>>> Having an extra repo with its own set of branches is cheap and easy -
>>>>> we already do that with dtests. I like cleanly separated things when
>>>>> coupling is avoidable. As such I would prefer the sidecar to live in a
>>>>> separate new repo, while still being part of the C* project.
>>>>>>>
>>>>>>> —
>>>>>>> AY
>>>>>>>
>>>>>>> On 21 August 2018 at 17:06:39, sankalp kohli (kohlisankalp@gmail.com
>> )
>>>>> wrote:
>>>>>>>
>>>>>>> Hi Aleksey,
>>>>>>> Can you please elaborate on the reasons for your -1? This
>>>>>>> way we can make progress towards any one approach.
>>>>>>> Thanks,
>>>>>>> Sankalp
>>>>>>>
>>>>>>> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <
>> aleksey@apple.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> FWIW I’m strongly -1 on in-tree approach, and would much prefer a
>>>>> separate
>>>>>>>> repo, dtest-style.
>>>>>>>>
>>>>>>>> —
>>>>>>>> AY
>>>>>>>>
>>>>>>>> On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
>>>>>>>> jeremiah.jordan@gmail.com) wrote:
>>>>>>>>
>>>>>>>> I think the following is a very big plus of it being in tree:
>>>>>>>>>> * Faster iteration speed in general. For example when we need to
>>>>> add a
>>>>>>>>>> new
>>>>>>>>>> JMX endpoint that the sidecar needs, or change something from
>>>>> JMX to a
>>>>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
>>>>> changes
>>>>>>>>>> including tests as one commit within the main repository and
>>>>> don't
>>>>>>>> have
>>>>>>>>>> to
>>>>>>>>>> commit to main repo, sidecar repo,
>>>>>>>>
>>>>>>>> I also don’t see a reason why the sidecar being in tree means it
>>>>> would not
>>>>>>>> work in a mixed version cluster. The nodes themselves must work in a
>>>>> mixed
>>>>>>>> version cluster during a rolling upgrade, I would expect any
>>>>> management
>>>>>>>> side car to operate in the same manor, in tree or not.
>>>>>>>>
>>>>>>>> This tool will be pretty tightly coupled with the server, and as
>>>>> someone
>>>>>>>> with experience developing such tightly coupled tools, it is *much*
>>>>> easier
>>>>>>>> to make sure you don’t accidentally break them if they are in tree.
>>>>> How
>>>>>>>> many times has someone updated some JMX interface, updated nodetool,
>>>>> and
>>>>>>>> then moved on? Breaking all the external tools not in tree, without
>>>>>>>> realizing it. The above point about being able to modify interfaces
>>>>> and the
>>>>>>>> side car in the same commit is huge in terms of making sure someone
>>>>> doesn’t
>>>>>>>> inadvertently break the side car while fixing something else.
>>>>>>>>
>>>>>>>> -Jeremiah
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Strongly agree with Blake. In my mind supporting multiple versions
>>>>> is
>>>>>>>>> mandatory. As I've stated before, we already do it with Reaper, I'd
>>>>>>>>> consider it a major misstep if we couldn't support multiple with
>>>>> the
>>>>>>>>> project - provided admin tool. It's the same reason dtests are
>>>>> separate
>>>>>>>> -
>>>>>>>>> they work with multiple versions.
>>>>>>>>>
>>>>>>>>> The number of repos does not affect distribution - if we want to
>>>>> ship
>>>>>>>>> Cassandra with the admin / repair tool (we should, imo), that can
>>>>> be
>>>>>>>> part
>>>>>>>>> of the build process.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <
>>>>> beggleston@apple.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> If the sidecar is going to be on a different release cadence, or
>>>>>>>> support
>>>>>>>>>> interacting with mixed mode clusters, then it should definitely
>>>>> be in
>>>>>>>> a
>>>>>>>>>> separate repo. I don’t even know how branching and merging would
>>>>> work
>>>>>>>> in a
>>>>>>>>>> repo that supports 2 separate release targets and/or mixed mode
>>>>>>>>>> compatibility, but I’m pretty sure it would be a mess.
>>>>>>>>>>
>>>>>>>>>> As a cluster management tool, mixed mode is probably going to be
>>>>> a goal
>>>>>>>> at
>>>>>>>>>> some point. As a new project, it will benefit from not being
>>>>> tied to
>>>>>>>> the C*
>>>>>>>>>> release cycle (which would probably delay any sidecar release
>>>>> until
>>>>>>>>>> whenever 4.1 is cut).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (
>>>>> joe.e.lynch@gmail.com)
>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I think that the pros of incubating the sidecar in tree as a tool
>>>>>>>> first
>>>>>>>>>> outweigh the alternatives at this point of time. Rough tradeoffs
>>>>> that
>>>>>>>> I
>>>>>>>>>> see:
>>>>>>>>>>
>>>>>>>>>> Unique pros of in tree sidecar:
>>>>>>>>>> * Faster iteration speed in general. For example when we need to
>>>>> add a
>>>>>>>>>> new
>>>>>>>>>> JMX endpoint that the sidecar needs, or change something from
>>>>> JMX to a
>>>>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
>>>>> changes
>>>>>>>>>> including tests as one commit within the main repository and
>>>>> don't
>>>>>>>> have
>>>>>>>>>> to
>>>>>>>>>> commit to main repo, sidecar repo, and dtest repo (juggling
>>>>> version
>>>>>>>>>> compatibility along the way).
>>>>>>>>>> * We can in the future more easily move serious background
>>>>>>>> functionality
>>>>>>>>>> like compaction or repair itself (not repair scheduling, actual
>>>>>>>>>> repairing)
>>>>>>>>>> into the sidecar with a single atomic commit, we don't have to
>>>>> do two
>>>>>>>>>> phase
>>>>>>>>>> commits where we add some IPC mechanism to allow us to support
>>>>> it in
>>>>>>>>>> both,
>>>>>>>>>> then turn it on in the sidecar, then turn it off in the server,
>>>>> etc...
>>>>>>>>>> * I think that the verification is much easier (sounds like
>>>>> Jonathan
>>>>>>>>>> disagreed on the other thread, I could certainly be wrong), and
>>>>> we
>>>>>>>> don't
>>>>>>>>>> have to worry about testing matrices to assure that the sidecar
>>>>> works
>>>>>>>>>> with
>>>>>>>>>> various versions as the version of the sidecar that is released
>>>>> with
>>>>>>>> that
>>>>>>>>>> version of Cassandra is the only one we have to certify works. If
>>>>>>>> people
>>>>>>>>>> want to pull in new versions or maintain backports they can do
>>>>> that at
>>>>>>>>>> their discretion/testing.
>>>>>>>>>> * We can iterate and prove value before committing to a choice.
>>>>> Since
>>>>>>>> it
>>>>>>>>>> will be a separate artifact from the start we can always move the
>>>>>>>>>> artifact
>>>>>>>>>> to a separate repo later (but moving the other way is harder).
>>>>>>>>>> * Users will get the sidecar "for free" when they install the
>>>>> daemon,
>>>>>>>>>> they
>>>>>>>>>> don't need to take affirmative action to e.g. be able to restart
>>>>> their
>>>>>>>>>> cluster, run repair, or back their data up; it just comes out of
>>>>> the
>>>>>>>> box
>>>>>>>>>> for free.
>>>>>>>>>>
>>>>>>>>>> Unique pros of a separate repository sidecar:
>>>>>>>>>> * We can use a more modern build system like gradle instead of
>>>>> ant
>>>>>>>>>> * Merging changes is less "scary" I guess (I feel like if you're
>>>>> not
>>>>>>>>>> touching the daemon this is already true but I could see this
>>>>> being
>>>>>>>> less
>>>>>>>>>> worrisome for some).
>>>>>>>>>> * Releasing a separate artifact is somewhat easier from a
>>>>> separate
>>>>>>>> repo
>>>>>>>>>> (especially if we have gradle which makes e.g. building debs and
>>>>> rpms
>>>>>>>>>> trivial).
>>>>>>>>>> * We could backport to previous versions without getting into
>>>>>>>> arguments
>>>>>>>>>> about bug fixes vs features.
>>>>>>>>>> * Committers could be different from the main repo, which ...
>>>>> may be a
>>>>>>>>>> useful thing
>>>>>>>>>>
>>>>>>>>>> Non unique pros of a sidecar (could be achieved in the main repo
>>>>> or in
>>>>>>>> a
>>>>>>>>>> separate repo):
>>>>>>>>>> * A separate build artifact .jar/.deb/.rpm that can be installed
>>>>>>>>>> separately. It's slightly easier with a separate repo but
>>>>> certainly
>>>>>>>> not
>>>>>>>>>> out
>>>>>>>>>> of reach within a single repo (indeed the current patch already
>>>>> creates
>>>>>>>> a
>>>>>>>>>> separate jar, and we could create a separate .deb reasonably
>>>>> easily).
>>>>>>>>>> Personally I think having a separate .deb/.rpm is premature at
>>>>> this
>>>>>>>> point
>>>>>>>>>> (for companies that really want it they can build their own
>>>>> packages
>>>>>>>>>> using
>>>>>>>>>> the .jars), but I think it really is a distracting issue from
>>>>> where
>>>>>>>> the
>>>>>>>>>> patch should go as we can always choose to remove experimental
>>>>> .jar
>>>>>>>> files
>>>>>>>>>> that the main daemon doesn't touch.
>>>>>>>>>> * A separate process lifecycle. No matter where the sidecar
>>>>> goes, we
>>>>>>>> get
>>>>>>>>>> the benefit of restarting it being less dangerous for
>>>>> availability
>>>>>>>> than
>>>>>>>>>> restarting the main daemon.
>>>>>>>>>>
>>>>>>>>>> That all being said, these are strong opinions weakly held and I
>>>>> would
>>>>>>>>>> rather get something actually committed so that we can prove
>>>>> value one
>>>>>>>>>> way
>>>>>>>>>> or the other and am therefore, of course, happy to put sidecar
>>>>> patches
>>>>>>>>>> wherever someone can review and commit it.
>>>>>>>>>>
>>>>>>>>>> -Joey
>>>>>>>>>>
>>>>>>>>>> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <
>>>>> kohlisankalp@gmail.com>
>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>> I am starting a new thread to get consensus on where the side
>>>>> car
>>>>>>>>>>> should be contributed.
>>>>>>>>>>>
>>>>>>>>>>> Please send your responses with pro/cons of each approach or
>>>>> any
>>>>>>>> other
>>>>>>>>>>> approach. Please be clear which approach you will pick while
>>>>> still
>>>>>>>>>> giving
>>>>>>>>>>> pros/cons of both approaches.
>>>>>>>>>>>
>>>>>>>>>>> Thanks.
>>>>>>>>>>> Sankalp
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Jon Haddad
>>>>>>>>> http://www.rustyrazorblade.com
>>>>>>>>> twitter: rustyrazorblade
>>>>>>>>
>>>>>>>>
>>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>>>
>>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>
>> --
> 
> Muru
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org

Re: Side Car New Repo vs not

[Murukesh Mohanan <mu...@gmail.com>]

FWIW, I think it's possible to merge in a separate repository into a
subdirectory while keeping git history, but I don't know if the other way
will be possible if commits span other parts of the repo as well\* (which
will likely happen sooner or later). So a separate repo is a choice we can
backtrack from if it proves problematic later.


\* it may be possible, but the commit messages might not make much sense
after that.

On Fri, 24 Aug 2018, 09:12 Benedict Elliott Smith, <be...@apache.org>
wrote:

> +1 also for separate repo
>
> > On 24 Aug 2018, at 01:11, Jeff Jirsa <jj...@gmail.com> wrote:
> >
> > +1 for separate repo
> >
> >
> > --
> > Jeff Jirsa
> >
> >
> >> On Aug 23, 2018, at 1:00 PM, sankalp kohli <ko...@gmail.com>
> wrote:
> >>
> >> Separate repo is in a majority so far. Please reply to this thread with
> >> your responses.
> >>
> >> On Tue, Aug 21, 2018 at 4:34 PM Rahul Singh <
> rahul.xavier.singh@gmail.com>
> >> wrote:
> >>
> >>> +1 for separate repo. Especially on git. Maybe make it a submodule.
> >>>
> >>> Rahul
> >>> On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski <sp...@apache.org>,
> >>> wrote:
> >>>> I'm also currently -1 on the in-tree option.
> >>>>
> >>>> Additionally to what Aleksey mentioned, I also don't see how we could
> >>>> make this work with the current build and release process. Our scripts
> >>>> [0] for creating releases (tarballs and native packages), would need
> >>>> significant work to add support for an independent side-car. Our ant
> >>>> based build process is also not a great start for adding new tasks,
> let
> >>>> alone integrating other tool chains for web components for a potential
> >>> UI.
> >>>>
> >>>> [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git
> >>>>
> >>>>
> >>>>> On 21.08.18 19:20, Aleksey Yeshchenko wrote:
> >>>>> Sure, allow me to elaborate - at least a little bit. But before I do,
> >>> just let me note that this wasn’t a veto -1, just a shorthand for “I
> don’t
> >>> like this option”.
> >>>>>
> >>>>> It would be nice to have sidecar and C* version and release cycles
> >>> fully decoupled. I know it *can* be done when in-tree, but the way we
> vote
> >>> on releases with tags off current branches would have to change
> somehow.
> >>> Probably painfully. It would be nice to be able to easily enforce
> freezes,
> >>> like the upcoming one, on the whole C* repo, while allowing feature
> >>> development on the sidecar. It would be nice to not have sidecar
> commits in
> >>> emails from commits@ mailing list. It would be nice to not have C* CI
> >>> trigger necessarily on sidecar commits. Groups of people working on
> the two
> >>> repos will mostly be different too, so what’s the point in sharing the
> repo?
> >>>>>
> >>>>> Having an extra repo with its own set of branches is cheap and easy -
> >>> we already do that with dtests. I like cleanly separated things when
> >>> coupling is avoidable. As such I would prefer the sidecar to live in a
> >>> separate new repo, while still being part of the C* project.
> >>>>>
> >>>>> —
> >>>>> AY
> >>>>>
> >>>>> On 21 August 2018 at 17:06:39, sankalp kohli (kohlisankalp@gmail.com
> )
> >>> wrote:
> >>>>>
> >>>>> Hi Aleksey,
> >>>>> Can you please elaborate on the reasons for your -1? This
> >>>>> way we can make progress towards any one approach.
> >>>>> Thanks,
> >>>>> Sankalp
> >>>>>
> >>>>> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <
> aleksey@apple.com>
> >>>>> wrote:
> >>>>>
> >>>>>> FWIW I’m strongly -1 on in-tree approach, and would much prefer a
> >>> separate
> >>>>>> repo, dtest-style.
> >>>>>>
> >>>>>> —
> >>>>>> AY
> >>>>>>
> >>>>>> On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> >>>>>> jeremiah.jordan@gmail.com) wrote:
> >>>>>>
> >>>>>> I think the following is a very big plus of it being in tree:
> >>>>>>>> * Faster iteration speed in general. For example when we need to
> >>> add a
> >>>>>>>> new
> >>>>>>>> JMX endpoint that the sidecar needs, or change something from
> >>> JMX to a
> >>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
> >>> changes
> >>>>>>>> including tests as one commit within the main repository and
> >>> don't
> >>>>>> have
> >>>>>>>> to
> >>>>>>>> commit to main repo, sidecar repo,
> >>>>>>
> >>>>>> I also don’t see a reason why the sidecar being in tree means it
> >>> would not
> >>>>>> work in a mixed version cluster. The nodes themselves must work in a
> >>> mixed
> >>>>>> version cluster during a rolling upgrade, I would expect any
> >>> management
> >>>>>> side car to operate in the same manor, in tree or not.
> >>>>>>
> >>>>>> This tool will be pretty tightly coupled with the server, and as
> >>> someone
> >>>>>> with experience developing such tightly coupled tools, it is *much*
> >>> easier
> >>>>>> to make sure you don’t accidentally break them if they are in tree.
> >>> How
> >>>>>> many times has someone updated some JMX interface, updated nodetool,
> >>> and
> >>>>>> then moved on? Breaking all the external tools not in tree, without
> >>>>>> realizing it. The above point about being able to modify interfaces
> >>> and the
> >>>>>> side car in the same commit is huge in terms of making sure someone
> >>> doesn’t
> >>>>>> inadvertently break the side car while fixing something else.
> >>>>>>
> >>>>>> -Jeremiah
> >>>>>>
> >>>>>>
> >>>>>>> On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Strongly agree with Blake. In my mind supporting multiple versions
> >>> is
> >>>>>>> mandatory. As I've stated before, we already do it with Reaper, I'd
> >>>>>>> consider it a major misstep if we couldn't support multiple with
> >>> the
> >>>>>>> project - provided admin tool. It's the same reason dtests are
> >>> separate
> >>>>>> -
> >>>>>>> they work with multiple versions.
> >>>>>>>
> >>>>>>> The number of repos does not affect distribution - if we want to
> >>> ship
> >>>>>>> Cassandra with the admin / repair tool (we should, imo), that can
> >>> be
> >>>>>> part
> >>>>>>> of the build process.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <
> >>> beggleston@apple.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> If the sidecar is going to be on a different release cadence, or
> >>>>>> support
> >>>>>>>> interacting with mixed mode clusters, then it should definitely
> >>> be in
> >>>>>> a
> >>>>>>>> separate repo. I don’t even know how branching and merging would
> >>> work
> >>>>>> in a
> >>>>>>>> repo that supports 2 separate release targets and/or mixed mode
> >>>>>>>> compatibility, but I’m pretty sure it would be a mess.
> >>>>>>>>
> >>>>>>>> As a cluster management tool, mixed mode is probably going to be
> >>> a goal
> >>>>>> at
> >>>>>>>> some point. As a new project, it will benefit from not being
> >>> tied to
> >>>>>> the C*
> >>>>>>>> release cycle (which would probably delay any sidecar release
> >>> until
> >>>>>>>> whenever 4.1 is cut).
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (
> >>> joe.e.lynch@gmail.com)
> >>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> I think that the pros of incubating the sidecar in tree as a tool
> >>>>>> first
> >>>>>>>> outweigh the alternatives at this point of time. Rough tradeoffs
> >>> that
> >>>>>> I
> >>>>>>>> see:
> >>>>>>>>
> >>>>>>>> Unique pros of in tree sidecar:
> >>>>>>>> * Faster iteration speed in general. For example when we need to
> >>> add a
> >>>>>>>> new
> >>>>>>>> JMX endpoint that the sidecar needs, or change something from
> >>> JMX to a
> >>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
> >>> changes
> >>>>>>>> including tests as one commit within the main repository and
> >>> don't
> >>>>>> have
> >>>>>>>> to
> >>>>>>>> commit to main repo, sidecar repo, and dtest repo (juggling
> >>> version
> >>>>>>>> compatibility along the way).
> >>>>>>>> * We can in the future more easily move serious background
> >>>>>> functionality
> >>>>>>>> like compaction or repair itself (not repair scheduling, actual
> >>>>>>>> repairing)
> >>>>>>>> into the sidecar with a single atomic commit, we don't have to
> >>> do two
> >>>>>>>> phase
> >>>>>>>> commits where we add some IPC mechanism to allow us to support
> >>> it in
> >>>>>>>> both,
> >>>>>>>> then turn it on in the sidecar, then turn it off in the server,
> >>> etc...
> >>>>>>>> * I think that the verification is much easier (sounds like
> >>> Jonathan
> >>>>>>>> disagreed on the other thread, I could certainly be wrong), and
> >>> we
> >>>>>> don't
> >>>>>>>> have to worry about testing matrices to assure that the sidecar
> >>> works
> >>>>>>>> with
> >>>>>>>> various versions as the version of the sidecar that is released
> >>> with
> >>>>>> that
> >>>>>>>> version of Cassandra is the only one we have to certify works. If
> >>>>>> people
> >>>>>>>> want to pull in new versions or maintain backports they can do
> >>> that at
> >>>>>>>> their discretion/testing.
> >>>>>>>> * We can iterate and prove value before committing to a choice.
> >>> Since
> >>>>>> it
> >>>>>>>> will be a separate artifact from the start we can always move the
> >>>>>>>> artifact
> >>>>>>>> to a separate repo later (but moving the other way is harder).
> >>>>>>>> * Users will get the sidecar "for free" when they install the
> >>> daemon,
> >>>>>>>> they
> >>>>>>>> don't need to take affirmative action to e.g. be able to restart
> >>> their
> >>>>>>>> cluster, run repair, or back their data up; it just comes out of
> >>> the
> >>>>>> box
> >>>>>>>> for free.
> >>>>>>>>
> >>>>>>>> Unique pros of a separate repository sidecar:
> >>>>>>>> * We can use a more modern build system like gradle instead of
> >>> ant
> >>>>>>>> * Merging changes is less "scary" I guess (I feel like if you're
> >>> not
> >>>>>>>> touching the daemon this is already true but I could see this
> >>> being
> >>>>>> less
> >>>>>>>> worrisome for some).
> >>>>>>>> * Releasing a separate artifact is somewhat easier from a
> >>> separate
> >>>>>> repo
> >>>>>>>> (especially if we have gradle which makes e.g. building debs and
> >>> rpms
> >>>>>>>> trivial).
> >>>>>>>> * We could backport to previous versions without getting into
> >>>>>> arguments
> >>>>>>>> about bug fixes vs features.
> >>>>>>>> * Committers could be different from the main repo, which ...
> >>> may be a
> >>>>>>>> useful thing
> >>>>>>>>
> >>>>>>>> Non unique pros of a sidecar (could be achieved in the main repo
> >>> or in
> >>>>>> a
> >>>>>>>> separate repo):
> >>>>>>>> * A separate build artifact .jar/.deb/.rpm that can be installed
> >>>>>>>> separately. It's slightly easier with a separate repo but
> >>> certainly
> >>>>>> not
> >>>>>>>> out
> >>>>>>>> of reach within a single repo (indeed the current patch already
> >>> creates
> >>>>>> a
> >>>>>>>> separate jar, and we could create a separate .deb reasonably
> >>> easily).
> >>>>>>>> Personally I think having a separate .deb/.rpm is premature at
> >>> this
> >>>>>> point
> >>>>>>>> (for companies that really want it they can build their own
> >>> packages
> >>>>>>>> using
> >>>>>>>> the .jars), but I think it really is a distracting issue from
> >>> where
> >>>>>> the
> >>>>>>>> patch should go as we can always choose to remove experimental
> >>> .jar
> >>>>>> files
> >>>>>>>> that the main daemon doesn't touch.
> >>>>>>>> * A separate process lifecycle. No matter where the sidecar
> >>> goes, we
> >>>>>> get
> >>>>>>>> the benefit of restarting it being less dangerous for
> >>> availability
> >>>>>> than
> >>>>>>>> restarting the main daemon.
> >>>>>>>>
> >>>>>>>> That all being said, these are strong opinions weakly held and I
> >>> would
> >>>>>>>> rather get something actually committed so that we can prove
> >>> value one
> >>>>>>>> way
> >>>>>>>> or the other and am therefore, of course, happy to put sidecar
> >>> patches
> >>>>>>>> wherever someone can review and commit it.
> >>>>>>>>
> >>>>>>>> -Joey
> >>>>>>>>
> >>>>>>>> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <
> >>> kohlisankalp@gmail.com>
> >>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>> I am starting a new thread to get consensus on where the side
> >>> car
> >>>>>>>>> should be contributed.
> >>>>>>>>>
> >>>>>>>>> Please send your responses with pro/cons of each approach or
> >>> any
> >>>>>> other
> >>>>>>>>> approach. Please be clear which approach you will pick while
> >>> still
> >>>>>>>> giving
> >>>>>>>>> pros/cons of both approaches.
> >>>>>>>>>
> >>>>>>>>> Thanks.
> >>>>>>>>> Sankalp
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Jon Haddad
> >>>>>>> http://www.rustyrazorblade.com
> >>>>>>> twitter: rustyrazorblade
> >>>>>>
> >>>>>>
> >>>>>>
> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>
> >>>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
> --

Muru

Re: Side Car New Repo vs not

[Benedict Elliott Smith <be...@apache.org>]

+1 also for separate repo

> On 24 Aug 2018, at 01:11, Jeff Jirsa <jj...@gmail.com> wrote:
> 
> +1 for separate repo
> 
> 
> -- 
> Jeff Jirsa
> 
> 
>> On Aug 23, 2018, at 1:00 PM, sankalp kohli <ko...@gmail.com> wrote:
>> 
>> Separate repo is in a majority so far. Please reply to this thread with
>> your responses.
>> 
>> On Tue, Aug 21, 2018 at 4:34 PM Rahul Singh <ra...@gmail.com>
>> wrote:
>> 
>>> +1 for separate repo. Especially on git. Maybe make it a submodule.
>>> 
>>> Rahul
>>> On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski <sp...@apache.org>,
>>> wrote:
>>>> I'm also currently -1 on the in-tree option.
>>>> 
>>>> Additionally to what Aleksey mentioned, I also don't see how we could
>>>> make this work with the current build and release process. Our scripts
>>>> [0] for creating releases (tarballs and native packages), would need
>>>> significant work to add support for an independent side-car. Our ant
>>>> based build process is also not a great start for adding new tasks, let
>>>> alone integrating other tool chains for web components for a potential
>>> UI.
>>>> 
>>>> [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git
>>>> 
>>>> 
>>>>> On 21.08.18 19:20, Aleksey Yeshchenko wrote:
>>>>> Sure, allow me to elaborate - at least a little bit. But before I do,
>>> just let me note that this wasn’t a veto -1, just a shorthand for “I don’t
>>> like this option”.
>>>>> 
>>>>> It would be nice to have sidecar and C* version and release cycles
>>> fully decoupled. I know it *can* be done when in-tree, but the way we vote
>>> on releases with tags off current branches would have to change somehow.
>>> Probably painfully. It would be nice to be able to easily enforce freezes,
>>> like the upcoming one, on the whole C* repo, while allowing feature
>>> development on the sidecar. It would be nice to not have sidecar commits in
>>> emails from commits@ mailing list. It would be nice to not have C* CI
>>> trigger necessarily on sidecar commits. Groups of people working on the two
>>> repos will mostly be different too, so what’s the point in sharing the repo?
>>>>> 
>>>>> Having an extra repo with its own set of branches is cheap and easy -
>>> we already do that with dtests. I like cleanly separated things when
>>> coupling is avoidable. As such I would prefer the sidecar to live in a
>>> separate new repo, while still being part of the C* project.
>>>>> 
>>>>> —
>>>>> AY
>>>>> 
>>>>> On 21 August 2018 at 17:06:39, sankalp kohli (kohlisankalp@gmail.com)
>>> wrote:
>>>>> 
>>>>> Hi Aleksey,
>>>>> Can you please elaborate on the reasons for your -1? This
>>>>> way we can make progress towards any one approach.
>>>>> Thanks,
>>>>> Sankalp
>>>>> 
>>>>> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <al...@apple.com>
>>>>> wrote:
>>>>> 
>>>>>> FWIW I’m strongly -1 on in-tree approach, and would much prefer a
>>> separate
>>>>>> repo, dtest-style.
>>>>>> 
>>>>>> —
>>>>>> AY
>>>>>> 
>>>>>> On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
>>>>>> jeremiah.jordan@gmail.com) wrote:
>>>>>> 
>>>>>> I think the following is a very big plus of it being in tree:
>>>>>>>> * Faster iteration speed in general. For example when we need to
>>> add a
>>>>>>>> new
>>>>>>>> JMX endpoint that the sidecar needs, or change something from
>>> JMX to a
>>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
>>> changes
>>>>>>>> including tests as one commit within the main repository and
>>> don't
>>>>>> have
>>>>>>>> to
>>>>>>>> commit to main repo, sidecar repo,
>>>>>> 
>>>>>> I also don’t see a reason why the sidecar being in tree means it
>>> would not
>>>>>> work in a mixed version cluster. The nodes themselves must work in a
>>> mixed
>>>>>> version cluster during a rolling upgrade, I would expect any
>>> management
>>>>>> side car to operate in the same manor, in tree or not.
>>>>>> 
>>>>>> This tool will be pretty tightly coupled with the server, and as
>>> someone
>>>>>> with experience developing such tightly coupled tools, it is *much*
>>> easier
>>>>>> to make sure you don’t accidentally break them if they are in tree.
>>> How
>>>>>> many times has someone updated some JMX interface, updated nodetool,
>>> and
>>>>>> then moved on? Breaking all the external tools not in tree, without
>>>>>> realizing it. The above point about being able to modify interfaces
>>> and the
>>>>>> side car in the same commit is huge in terms of making sure someone
>>> doesn’t
>>>>>> inadvertently break the side car while fixing something else.
>>>>>> 
>>>>>> -Jeremiah
>>>>>> 
>>>>>> 
>>>>>>> On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Strongly agree with Blake. In my mind supporting multiple versions
>>> is
>>>>>>> mandatory. As I've stated before, we already do it with Reaper, I'd
>>>>>>> consider it a major misstep if we couldn't support multiple with
>>> the
>>>>>>> project - provided admin tool. It's the same reason dtests are
>>> separate
>>>>>> -
>>>>>>> they work with multiple versions.
>>>>>>> 
>>>>>>> The number of repos does not affect distribution - if we want to
>>> ship
>>>>>>> Cassandra with the admin / repair tool (we should, imo), that can
>>> be
>>>>>> part
>>>>>>> of the build process.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <
>>> beggleston@apple.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> If the sidecar is going to be on a different release cadence, or
>>>>>> support
>>>>>>>> interacting with mixed mode clusters, then it should definitely
>>> be in
>>>>>> a
>>>>>>>> separate repo. I don’t even know how branching and merging would
>>> work
>>>>>> in a
>>>>>>>> repo that supports 2 separate release targets and/or mixed mode
>>>>>>>> compatibility, but I’m pretty sure it would be a mess.
>>>>>>>> 
>>>>>>>> As a cluster management tool, mixed mode is probably going to be
>>> a goal
>>>>>> at
>>>>>>>> some point. As a new project, it will benefit from not being
>>> tied to
>>>>>> the C*
>>>>>>>> release cycle (which would probably delay any sidecar release
>>> until
>>>>>>>> whenever 4.1 is cut).
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (
>>> joe.e.lynch@gmail.com)
>>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> I think that the pros of incubating the sidecar in tree as a tool
>>>>>> first
>>>>>>>> outweigh the alternatives at this point of time. Rough tradeoffs
>>> that
>>>>>> I
>>>>>>>> see:
>>>>>>>> 
>>>>>>>> Unique pros of in tree sidecar:
>>>>>>>> * Faster iteration speed in general. For example when we need to
>>> add a
>>>>>>>> new
>>>>>>>> JMX endpoint that the sidecar needs, or change something from
>>> JMX to a
>>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
>>> changes
>>>>>>>> including tests as one commit within the main repository and
>>> don't
>>>>>> have
>>>>>>>> to
>>>>>>>> commit to main repo, sidecar repo, and dtest repo (juggling
>>> version
>>>>>>>> compatibility along the way).
>>>>>>>> * We can in the future more easily move serious background
>>>>>> functionality
>>>>>>>> like compaction or repair itself (not repair scheduling, actual
>>>>>>>> repairing)
>>>>>>>> into the sidecar with a single atomic commit, we don't have to
>>> do two
>>>>>>>> phase
>>>>>>>> commits where we add some IPC mechanism to allow us to support
>>> it in
>>>>>>>> both,
>>>>>>>> then turn it on in the sidecar, then turn it off in the server,
>>> etc...
>>>>>>>> * I think that the verification is much easier (sounds like
>>> Jonathan
>>>>>>>> disagreed on the other thread, I could certainly be wrong), and
>>> we
>>>>>> don't
>>>>>>>> have to worry about testing matrices to assure that the sidecar
>>> works
>>>>>>>> with
>>>>>>>> various versions as the version of the sidecar that is released
>>> with
>>>>>> that
>>>>>>>> version of Cassandra is the only one we have to certify works. If
>>>>>> people
>>>>>>>> want to pull in new versions or maintain backports they can do
>>> that at
>>>>>>>> their discretion/testing.
>>>>>>>> * We can iterate and prove value before committing to a choice.
>>> Since
>>>>>> it
>>>>>>>> will be a separate artifact from the start we can always move the
>>>>>>>> artifact
>>>>>>>> to a separate repo later (but moving the other way is harder).
>>>>>>>> * Users will get the sidecar "for free" when they install the
>>> daemon,
>>>>>>>> they
>>>>>>>> don't need to take affirmative action to e.g. be able to restart
>>> their
>>>>>>>> cluster, run repair, or back their data up; it just comes out of
>>> the
>>>>>> box
>>>>>>>> for free.
>>>>>>>> 
>>>>>>>> Unique pros of a separate repository sidecar:
>>>>>>>> * We can use a more modern build system like gradle instead of
>>> ant
>>>>>>>> * Merging changes is less "scary" I guess (I feel like if you're
>>> not
>>>>>>>> touching the daemon this is already true but I could see this
>>> being
>>>>>> less
>>>>>>>> worrisome for some).
>>>>>>>> * Releasing a separate artifact is somewhat easier from a
>>> separate
>>>>>> repo
>>>>>>>> (especially if we have gradle which makes e.g. building debs and
>>> rpms
>>>>>>>> trivial).
>>>>>>>> * We could backport to previous versions without getting into
>>>>>> arguments
>>>>>>>> about bug fixes vs features.
>>>>>>>> * Committers could be different from the main repo, which ...
>>> may be a
>>>>>>>> useful thing
>>>>>>>> 
>>>>>>>> Non unique pros of a sidecar (could be achieved in the main repo
>>> or in
>>>>>> a
>>>>>>>> separate repo):
>>>>>>>> * A separate build artifact .jar/.deb/.rpm that can be installed
>>>>>>>> separately. It's slightly easier with a separate repo but
>>> certainly
>>>>>> not
>>>>>>>> out
>>>>>>>> of reach within a single repo (indeed the current patch already
>>> creates
>>>>>> a
>>>>>>>> separate jar, and we could create a separate .deb reasonably
>>> easily).
>>>>>>>> Personally I think having a separate .deb/.rpm is premature at
>>> this
>>>>>> point
>>>>>>>> (for companies that really want it they can build their own
>>> packages
>>>>>>>> using
>>>>>>>> the .jars), but I think it really is a distracting issue from
>>> where
>>>>>> the
>>>>>>>> patch should go as we can always choose to remove experimental
>>> .jar
>>>>>> files
>>>>>>>> that the main daemon doesn't touch.
>>>>>>>> * A separate process lifecycle. No matter where the sidecar
>>> goes, we
>>>>>> get
>>>>>>>> the benefit of restarting it being less dangerous for
>>> availability
>>>>>> than
>>>>>>>> restarting the main daemon.
>>>>>>>> 
>>>>>>>> That all being said, these are strong opinions weakly held and I
>>> would
>>>>>>>> rather get something actually committed so that we can prove
>>> value one
>>>>>>>> way
>>>>>>>> or the other and am therefore, of course, happy to put sidecar
>>> patches
>>>>>>>> wherever someone can review and commit it.
>>>>>>>> 
>>>>>>>> -Joey
>>>>>>>> 
>>>>>>>> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <
>>> kohlisankalp@gmail.com>
>>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> I am starting a new thread to get consensus on where the side
>>> car
>>>>>>>>> should be contributed.
>>>>>>>>> 
>>>>>>>>> Please send your responses with pro/cons of each approach or
>>> any
>>>>>> other
>>>>>>>>> approach. Please be clear which approach you will pick while
>>> still
>>>>>>>> giving
>>>>>>>>> pros/cons of both approaches.
>>>>>>>>> 
>>>>>>>>> Thanks.
>>>>>>>>> Sankalp
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Jon Haddad
>>>>>>> http://www.rustyrazorblade.com
>>>>>>> twitter: rustyrazorblade
>>>>>> 
>>>>>> 
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>> 
>>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org

Re: Side Car New Repo vs not

[Jeff Jirsa <jj...@gmail.com>]

+1 for separate repo


-- 
Jeff Jirsa


> On Aug 23, 2018, at 1:00 PM, sankalp kohli <ko...@gmail.com> wrote:
> 
> Separate repo is in a majority so far. Please reply to this thread with
> your responses.
> 
> On Tue, Aug 21, 2018 at 4:34 PM Rahul Singh <ra...@gmail.com>
> wrote:
> 
>> +1 for separate repo. Especially on git. Maybe make it a submodule.
>> 
>> Rahul
>> On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski <sp...@apache.org>,
>> wrote:
>>> I'm also currently -1 on the in-tree option.
>>> 
>>> Additionally to what Aleksey mentioned, I also don't see how we could
>>> make this work with the current build and release process. Our scripts
>>> [0] for creating releases (tarballs and native packages), would need
>>> significant work to add support for an independent side-car. Our ant
>>> based build process is also not a great start for adding new tasks, let
>>> alone integrating other tool chains for web components for a potential
>> UI.
>>> 
>>> [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git
>>> 
>>> 
>>>> On 21.08.18 19:20, Aleksey Yeshchenko wrote:
>>>> Sure, allow me to elaborate - at least a little bit. But before I do,
>> just let me note that this wasn’t a veto -1, just a shorthand for “I don’t
>> like this option”.
>>>> 
>>>> It would be nice to have sidecar and C* version and release cycles
>> fully decoupled. I know it *can* be done when in-tree, but the way we vote
>> on releases with tags off current branches would have to change somehow.
>> Probably painfully. It would be nice to be able to easily enforce freezes,
>> like the upcoming one, on the whole C* repo, while allowing feature
>> development on the sidecar. It would be nice to not have sidecar commits in
>> emails from commits@ mailing list. It would be nice to not have C* CI
>> trigger necessarily on sidecar commits. Groups of people working on the two
>> repos will mostly be different too, so what’s the point in sharing the repo?
>>>> 
>>>> Having an extra repo with its own set of branches is cheap and easy -
>> we already do that with dtests. I like cleanly separated things when
>> coupling is avoidable. As such I would prefer the sidecar to live in a
>> separate new repo, while still being part of the C* project.
>>>> 
>>>> —
>>>> AY
>>>> 
>>>> On 21 August 2018 at 17:06:39, sankalp kohli (kohlisankalp@gmail.com)
>> wrote:
>>>> 
>>>> Hi Aleksey,
>>>> Can you please elaborate on the reasons for your -1? This
>>>> way we can make progress towards any one approach.
>>>> Thanks,
>>>> Sankalp
>>>> 
>>>> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <al...@apple.com>
>>>> wrote:
>>>> 
>>>>> FWIW I’m strongly -1 on in-tree approach, and would much prefer a
>> separate
>>>>> repo, dtest-style.
>>>>> 
>>>>> —
>>>>> AY
>>>>> 
>>>>> On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
>>>>> jeremiah.jordan@gmail.com) wrote:
>>>>> 
>>>>> I think the following is a very big plus of it being in tree:
>>>>>>> * Faster iteration speed in general. For example when we need to
>> add a
>>>>>>> new
>>>>>>> JMX endpoint that the sidecar needs, or change something from
>> JMX to a
>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
>> changes
>>>>>>> including tests as one commit within the main repository and
>> don't
>>>>> have
>>>>>>> to
>>>>>>> commit to main repo, sidecar repo,
>>>>> 
>>>>> I also don’t see a reason why the sidecar being in tree means it
>> would not
>>>>> work in a mixed version cluster. The nodes themselves must work in a
>> mixed
>>>>> version cluster during a rolling upgrade, I would expect any
>> management
>>>>> side car to operate in the same manor, in tree or not.
>>>>> 
>>>>> This tool will be pretty tightly coupled with the server, and as
>> someone
>>>>> with experience developing such tightly coupled tools, it is *much*
>> easier
>>>>> to make sure you don’t accidentally break them if they are in tree.
>> How
>>>>> many times has someone updated some JMX interface, updated nodetool,
>> and
>>>>> then moved on? Breaking all the external tools not in tree, without
>>>>> realizing it. The above point about being able to modify interfaces
>> and the
>>>>> side car in the same commit is huge in terms of making sure someone
>> doesn’t
>>>>> inadvertently break the side car while fixing something else.
>>>>> 
>>>>> -Jeremiah
>>>>> 
>>>>> 
>>>>>> On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>
>>>>> wrote:
>>>>>> 
>>>>>> Strongly agree with Blake. In my mind supporting multiple versions
>> is
>>>>>> mandatory. As I've stated before, we already do it with Reaper, I'd
>>>>>> consider it a major misstep if we couldn't support multiple with
>> the
>>>>>> project - provided admin tool. It's the same reason dtests are
>> separate
>>>>> -
>>>>>> they work with multiple versions.
>>>>>> 
>>>>>> The number of repos does not affect distribution - if we want to
>> ship
>>>>>> Cassandra with the admin / repair tool (we should, imo), that can
>> be
>>>>> part
>>>>>> of the build process.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <
>> beggleston@apple.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> If the sidecar is going to be on a different release cadence, or
>>>>> support
>>>>>>> interacting with mixed mode clusters, then it should definitely
>> be in
>>>>> a
>>>>>>> separate repo. I don’t even know how branching and merging would
>> work
>>>>> in a
>>>>>>> repo that supports 2 separate release targets and/or mixed mode
>>>>>>> compatibility, but I’m pretty sure it would be a mess.
>>>>>>> 
>>>>>>> As a cluster management tool, mixed mode is probably going to be
>> a goal
>>>>> at
>>>>>>> some point. As a new project, it will benefit from not being
>> tied to
>>>>> the C*
>>>>>>> release cycle (which would probably delay any sidecar release
>> until
>>>>>>> whenever 4.1 is cut).
>>>>>>> 
>>>>>>> 
>>>>>>> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (
>> joe.e.lynch@gmail.com)
>>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>> I think that the pros of incubating the sidecar in tree as a tool
>>>>> first
>>>>>>> outweigh the alternatives at this point of time. Rough tradeoffs
>> that
>>>>> I
>>>>>>> see:
>>>>>>> 
>>>>>>> Unique pros of in tree sidecar:
>>>>>>> * Faster iteration speed in general. For example when we need to
>> add a
>>>>>>> new
>>>>>>> JMX endpoint that the sidecar needs, or change something from
>> JMX to a
>>>>>>> virtual table (e.g. for repair, or monitoring) we can do all
>> changes
>>>>>>> including tests as one commit within the main repository and
>> don't
>>>>> have
>>>>>>> to
>>>>>>> commit to main repo, sidecar repo, and dtest repo (juggling
>> version
>>>>>>> compatibility along the way).
>>>>>>> * We can in the future more easily move serious background
>>>>> functionality
>>>>>>> like compaction or repair itself (not repair scheduling, actual
>>>>>>> repairing)
>>>>>>> into the sidecar with a single atomic commit, we don't have to
>> do two
>>>>>>> phase
>>>>>>> commits where we add some IPC mechanism to allow us to support
>> it in
>>>>>>> both,
>>>>>>> then turn it on in the sidecar, then turn it off in the server,
>> etc...
>>>>>>> * I think that the verification is much easier (sounds like
>> Jonathan
>>>>>>> disagreed on the other thread, I could certainly be wrong), and
>> we
>>>>> don't
>>>>>>> have to worry about testing matrices to assure that the sidecar
>> works
>>>>>>> with
>>>>>>> various versions as the version of the sidecar that is released
>> with
>>>>> that
>>>>>>> version of Cassandra is the only one we have to certify works. If
>>>>> people
>>>>>>> want to pull in new versions or maintain backports they can do
>> that at
>>>>>>> their discretion/testing.
>>>>>>> * We can iterate and prove value before committing to a choice.
>> Since
>>>>> it
>>>>>>> will be a separate artifact from the start we can always move the
>>>>>>> artifact
>>>>>>> to a separate repo later (but moving the other way is harder).
>>>>>>> * Users will get the sidecar "for free" when they install the
>> daemon,
>>>>>>> they
>>>>>>> don't need to take affirmative action to e.g. be able to restart
>> their
>>>>>>> cluster, run repair, or back their data up; it just comes out of
>> the
>>>>> box
>>>>>>> for free.
>>>>>>> 
>>>>>>> Unique pros of a separate repository sidecar:
>>>>>>> * We can use a more modern build system like gradle instead of
>> ant
>>>>>>> * Merging changes is less "scary" I guess (I feel like if you're
>> not
>>>>>>> touching the daemon this is already true but I could see this
>> being
>>>>> less
>>>>>>> worrisome for some).
>>>>>>> * Releasing a separate artifact is somewhat easier from a
>> separate
>>>>> repo
>>>>>>> (especially if we have gradle which makes e.g. building debs and
>> rpms
>>>>>>> trivial).
>>>>>>> * We could backport to previous versions without getting into
>>>>> arguments
>>>>>>> about bug fixes vs features.
>>>>>>> * Committers could be different from the main repo, which ...
>> may be a
>>>>>>> useful thing
>>>>>>> 
>>>>>>> Non unique pros of a sidecar (could be achieved in the main repo
>> or in
>>>>> a
>>>>>>> separate repo):
>>>>>>> * A separate build artifact .jar/.deb/.rpm that can be installed
>>>>>>> separately. It's slightly easier with a separate repo but
>> certainly
>>>>> not
>>>>>>> out
>>>>>>> of reach within a single repo (indeed the current patch already
>> creates
>>>>> a
>>>>>>> separate jar, and we could create a separate .deb reasonably
>> easily).
>>>>>>> Personally I think having a separate .deb/.rpm is premature at
>> this
>>>>> point
>>>>>>> (for companies that really want it they can build their own
>> packages
>>>>>>> using
>>>>>>> the .jars), but I think it really is a distracting issue from
>> where
>>>>> the
>>>>>>> patch should go as we can always choose to remove experimental
>> .jar
>>>>> files
>>>>>>> that the main daemon doesn't touch.
>>>>>>> * A separate process lifecycle. No matter where the sidecar
>> goes, we
>>>>> get
>>>>>>> the benefit of restarting it being less dangerous for
>> availability
>>>>> than
>>>>>>> restarting the main daemon.
>>>>>>> 
>>>>>>> That all being said, these are strong opinions weakly held and I
>> would
>>>>>>> rather get something actually committed so that we can prove
>> value one
>>>>>>> way
>>>>>>> or the other and am therefore, of course, happy to put sidecar
>> patches
>>>>>>> wherever someone can review and commit it.
>>>>>>> 
>>>>>>> -Joey
>>>>>>> 
>>>>>>> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <
>> kohlisankalp@gmail.com>
>>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hi,
>>>>>>>> I am starting a new thread to get consensus on where the side
>> car
>>>>>>>> should be contributed.
>>>>>>>> 
>>>>>>>> Please send your responses with pro/cons of each approach or
>> any
>>>>> other
>>>>>>>> approach. Please be clear which approach you will pick while
>> still
>>>>>>> giving
>>>>>>>> pros/cons of both approaches.
>>>>>>>> 
>>>>>>>> Thanks.
>>>>>>>> Sankalp
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Jon Haddad
>>>>>> http://www.rustyrazorblade.com
>>>>>> twitter: rustyrazorblade
>>>>> 
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>> 
>>>>> 
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>> 
>> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org

Re: Side Car New Repo vs not

[sankalp kohli <ko...@gmail.com>]

Separate repo is in a majority so far. Please reply to this thread with
your responses.

On Tue, Aug 21, 2018 at 4:34 PM Rahul Singh <ra...@gmail.com>
wrote:

> +1 for separate repo. Especially on git. Maybe make it a submodule.
>
> Rahul
> On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski <sp...@apache.org>,
> wrote:
> > I'm also currently -1 on the in-tree option.
> >
> > Additionally to what Aleksey mentioned, I also don't see how we could
> > make this work with the current build and release process. Our scripts
> > [0] for creating releases (tarballs and native packages), would need
> > significant work to add support for an independent side-car. Our ant
> > based build process is also not a great start for adding new tasks, let
> > alone integrating other tool chains for web components for a potential
> UI.
> >
> > [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git
> >
> >
> > On 21.08.18 19:20, Aleksey Yeshchenko wrote:
> > > Sure, allow me to elaborate - at least a little bit. But before I do,
> just let me note that this wasn’t a veto -1, just a shorthand for “I don’t
> like this option”.
> > >
> > > It would be nice to have sidecar and C* version and release cycles
> fully decoupled. I know it *can* be done when in-tree, but the way we vote
> on releases with tags off current branches would have to change somehow.
> Probably painfully. It would be nice to be able to easily enforce freezes,
> like the upcoming one, on the whole C* repo, while allowing feature
> development on the sidecar. It would be nice to not have sidecar commits in
> emails from commits@ mailing list. It would be nice to not have C* CI
> trigger necessarily on sidecar commits. Groups of people working on the two
> repos will mostly be different too, so what’s the point in sharing the repo?
> > >
> > > Having an extra repo with its own set of branches is cheap and easy -
> we already do that with dtests. I like cleanly separated things when
> coupling is avoidable. As such I would prefer the sidecar to live in a
> separate new repo, while still being part of the C* project.
> > >
> > > —
> > > AY
> > >
> > > On 21 August 2018 at 17:06:39, sankalp kohli (kohlisankalp@gmail.com)
> wrote:
> > >
> > > Hi Aleksey,
> > > Can you please elaborate on the reasons for your -1? This
> > > way we can make progress towards any one approach.
> > > Thanks,
> > > Sankalp
> > >
> > > On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <al...@apple.com>
> > > wrote:
> > >
> > > > FWIW I’m strongly -1 on in-tree approach, and would much prefer a
> separate
> > > > repo, dtest-style.
> > > >
> > > > —
> > > > AY
> > > >
> > > > On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> > > > jeremiah.jordan@gmail.com) wrote:
> > > >
> > > > I think the following is a very big plus of it being in tree:
> > > > > > * Faster iteration speed in general. For example when we need to
> add a
> > > > > > new
> > > > > > JMX endpoint that the sidecar needs, or change something from
> JMX to a
> > > > > > virtual table (e.g. for repair, or monitoring) we can do all
> changes
> > > > > > including tests as one commit within the main repository and
> don't
> > > > have
> > > > > > to
> > > > > > commit to main repo, sidecar repo,
> > > >
> > > > I also don’t see a reason why the sidecar being in tree means it
> would not
> > > > work in a mixed version cluster. The nodes themselves must work in a
> mixed
> > > > version cluster during a rolling upgrade, I would expect any
> management
> > > > side car to operate in the same manor, in tree or not.
> > > >
> > > > This tool will be pretty tightly coupled with the server, and as
> someone
> > > > with experience developing such tightly coupled tools, it is *much*
> easier
> > > > to make sure you don’t accidentally break them if they are in tree.
> How
> > > > many times has someone updated some JMX interface, updated nodetool,
> and
> > > > then moved on? Breaking all the external tools not in tree, without
> > > > realizing it. The above point about being able to modify interfaces
> and the
> > > > side car in the same commit is huge in terms of making sure someone
> doesn’t
> > > > inadvertently break the side car while fixing something else.
> > > >
> > > > -Jeremiah
> > > >
> > > >
> > > > > On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>
> > > > wrote:
> > > > >
> > > > > Strongly agree with Blake. In my mind supporting multiple versions
> is
> > > > > mandatory. As I've stated before, we already do it with Reaper, I'd
> > > > > consider it a major misstep if we couldn't support multiple with
> the
> > > > > project - provided admin tool. It's the same reason dtests are
> separate
> > > > -
> > > > > they work with multiple versions.
> > > > >
> > > > > The number of repos does not affect distribution - if we want to
> ship
> > > > > Cassandra with the admin / repair tool (we should, imo), that can
> be
> > > > part
> > > > > of the build process.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <
> beggleston@apple.com>
> > > > > wrote:
> > > > >
> > > > > > If the sidecar is going to be on a different release cadence, or
> > > > support
> > > > > > interacting with mixed mode clusters, then it should definitely
> be in
> > > > a
> > > > > > separate repo. I don’t even know how branching and merging would
> work
> > > > in a
> > > > > > repo that supports 2 separate release targets and/or mixed mode
> > > > > > compatibility, but I’m pretty sure it would be a mess.
> > > > > >
> > > > > > As a cluster management tool, mixed mode is probably going to be
> a goal
> > > > at
> > > > > > some point. As a new project, it will benefit from not being
> tied to
> > > > the C*
> > > > > > release cycle (which would probably delay any sidecar release
> until
> > > > > > whenever 4.1 is cut).
> > > > > >
> > > > > >
> > > > > > On August 20, 2018 at 3:22:54 PM, Joseph Lynch (
> joe.e.lynch@gmail.com)
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > I think that the pros of incubating the sidecar in tree as a tool
> > > > first
> > > > > > outweigh the alternatives at this point of time. Rough tradeoffs
> that
> > > > I
> > > > > > see:
> > > > > >
> > > > > > Unique pros of in tree sidecar:
> > > > > > * Faster iteration speed in general. For example when we need to
> add a
> > > > > > new
> > > > > > JMX endpoint that the sidecar needs, or change something from
> JMX to a
> > > > > > virtual table (e.g. for repair, or monitoring) we can do all
> changes
> > > > > > including tests as one commit within the main repository and
> don't
> > > > have
> > > > > > to
> > > > > > commit to main repo, sidecar repo, and dtest repo (juggling
> version
> > > > > > compatibility along the way).
> > > > > > * We can in the future more easily move serious background
> > > > functionality
> > > > > > like compaction or repair itself (not repair scheduling, actual
> > > > > > repairing)
> > > > > > into the sidecar with a single atomic commit, we don't have to
> do two
> > > > > > phase
> > > > > > commits where we add some IPC mechanism to allow us to support
> it in
> > > > > > both,
> > > > > > then turn it on in the sidecar, then turn it off in the server,
> etc...
> > > > > > * I think that the verification is much easier (sounds like
> Jonathan
> > > > > > disagreed on the other thread, I could certainly be wrong), and
> we
> > > > don't
> > > > > > have to worry about testing matrices to assure that the sidecar
> works
> > > > > > with
> > > > > > various versions as the version of the sidecar that is released
> with
> > > > that
> > > > > > version of Cassandra is the only one we have to certify works. If
> > > > people
> > > > > > want to pull in new versions or maintain backports they can do
> that at
> > > > > > their discretion/testing.
> > > > > > * We can iterate and prove value before committing to a choice.
> Since
> > > > it
> > > > > > will be a separate artifact from the start we can always move the
> > > > > > artifact
> > > > > > to a separate repo later (but moving the other way is harder).
> > > > > > * Users will get the sidecar "for free" when they install the
> daemon,
> > > > > > they
> > > > > > don't need to take affirmative action to e.g. be able to restart
> their
> > > > > > cluster, run repair, or back their data up; it just comes out of
> the
> > > > box
> > > > > > for free.
> > > > > >
> > > > > > Unique pros of a separate repository sidecar:
> > > > > > * We can use a more modern build system like gradle instead of
> ant
> > > > > > * Merging changes is less "scary" I guess (I feel like if you're
> not
> > > > > > touching the daemon this is already true but I could see this
> being
> > > > less
> > > > > > worrisome for some).
> > > > > > * Releasing a separate artifact is somewhat easier from a
> separate
> > > > repo
> > > > > > (especially if we have gradle which makes e.g. building debs and
> rpms
> > > > > > trivial).
> > > > > > * We could backport to previous versions without getting into
> > > > arguments
> > > > > > about bug fixes vs features.
> > > > > > * Committers could be different from the main repo, which ...
> may be a
> > > > > > useful thing
> > > > > >
> > > > > > Non unique pros of a sidecar (could be achieved in the main repo
> or in
> > > > a
> > > > > > separate repo):
> > > > > > * A separate build artifact .jar/.deb/.rpm that can be installed
> > > > > > separately. It's slightly easier with a separate repo but
> certainly
> > > > not
> > > > > > out
> > > > > > of reach within a single repo (indeed the current patch already
> creates
> > > > a
> > > > > > separate jar, and we could create a separate .deb reasonably
> easily).
> > > > > > Personally I think having a separate .deb/.rpm is premature at
> this
> > > > point
> > > > > > (for companies that really want it they can build their own
> packages
> > > > > > using
> > > > > > the .jars), but I think it really is a distracting issue from
> where
> > > > the
> > > > > > patch should go as we can always choose to remove experimental
> .jar
> > > > files
> > > > > > that the main daemon doesn't touch.
> > > > > > * A separate process lifecycle. No matter where the sidecar
> goes, we
> > > > get
> > > > > > the benefit of restarting it being less dangerous for
> availability
> > > > than
> > > > > > restarting the main daemon.
> > > > > >
> > > > > > That all being said, these are strong opinions weakly held and I
> would
> > > > > > rather get something actually committed so that we can prove
> value one
> > > > > > way
> > > > > > or the other and am therefore, of course, happy to put sidecar
> patches
> > > > > > wherever someone can review and commit it.
> > > > > >
> > > > > > -Joey
> > > > > >
> > > > > > On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <
> kohlisankalp@gmail.com>
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > > I am starting a new thread to get consensus on where the side
> car
> > > > > > > should be contributed.
> > > > > > >
> > > > > > > Please send your responses with pro/cons of each approach or
> any
> > > > other
> > > > > > > approach. Please be clear which approach you will pick while
> still
> > > > > > giving
> > > > > > > pros/cons of both approaches.
> > > > > > >
> > > > > > > Thanks.
> > > > > > > Sankalp
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Jon Haddad
> > > > > http://www.rustyrazorblade.com
> > > > > twitter: rustyrazorblade
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > > >
> > > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
>

Re: Side Car New Repo vs not

[Rahul Singh <ra...@gmail.com>]

+1 for separate repo. Especially on git. Maybe make it a submodule.

Rahul
On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski <sp...@apache.org>, wrote:
> I'm also currently -1 on the in-tree option.
>
> Additionally to what Aleksey mentioned, I also don't see how we could
> make this work with the current build and release process. Our scripts
> [0] for creating releases (tarballs and native packages), would need
> significant work to add support for an independent side-car. Our ant
> based build process is also not a great start for adding new tasks, let
> alone integrating other tool chains for web components for a potential UI.
>
> [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git
>
>
> On 21.08.18 19:20, Aleksey Yeshchenko wrote:
> > Sure, allow me to elaborate - at least a little bit. But before I do, just let me note that this wasn’t a veto -1, just a shorthand for “I don’t like this option”.
> >
> > It would be nice to have sidecar and C* version and release cycles fully decoupled. I know it *can* be done when in-tree, but the way we vote on releases with tags off current branches would have to change somehow. Probably painfully. It would be nice to be able to easily enforce freezes, like the upcoming one, on the whole C* repo, while allowing feature development on the sidecar. It would be nice to not have sidecar commits in emails from commits@ mailing list. It would be nice to not have C* CI trigger necessarily on sidecar commits. Groups of people working on the two repos will mostly be different too, so what’s the point in sharing the repo?
> >
> > Having an extra repo with its own set of branches is cheap and easy - we already do that with dtests. I like cleanly separated things when coupling is avoidable. As such I would prefer the sidecar to live in a separate new repo, while still being part of the C* project.
> >
> > —
> > AY
> >
> > On 21 August 2018 at 17:06:39, sankalp kohli (kohlisankalp@gmail.com) wrote:
> >
> > Hi Aleksey,
> > Can you please elaborate on the reasons for your -1? This
> > way we can make progress towards any one approach.
> > Thanks,
> > Sankalp
> >
> > On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <al...@apple.com>
> > wrote:
> >
> > > FWIW I’m strongly -1 on in-tree approach, and would much prefer a separate
> > > repo, dtest-style.
> > >
> > > —
> > > AY
> > >
> > > On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> > > jeremiah.jordan@gmail.com) wrote:
> > >
> > > I think the following is a very big plus of it being in tree:
> > > > > * Faster iteration speed in general. For example when we need to add a
> > > > > new
> > > > > JMX endpoint that the sidecar needs, or change something from JMX to a
> > > > > virtual table (e.g. for repair, or monitoring) we can do all changes
> > > > > including tests as one commit within the main repository and don't
> > > have
> > > > > to
> > > > > commit to main repo, sidecar repo,
> > >
> > > I also don’t see a reason why the sidecar being in tree means it would not
> > > work in a mixed version cluster. The nodes themselves must work in a mixed
> > > version cluster during a rolling upgrade, I would expect any management
> > > side car to operate in the same manor, in tree or not.
> > >
> > > This tool will be pretty tightly coupled with the server, and as someone
> > > with experience developing such tightly coupled tools, it is *much* easier
> > > to make sure you don’t accidentally break them if they are in tree. How
> > > many times has someone updated some JMX interface, updated nodetool, and
> > > then moved on? Breaking all the external tools not in tree, without
> > > realizing it. The above point about being able to modify interfaces and the
> > > side car in the same commit is huge in terms of making sure someone doesn’t
> > > inadvertently break the side car while fixing something else.
> > >
> > > -Jeremiah
> > >
> > >
> > > > On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>
> > > wrote:
> > > >
> > > > Strongly agree with Blake. In my mind supporting multiple versions is
> > > > mandatory. As I've stated before, we already do it with Reaper, I'd
> > > > consider it a major misstep if we couldn't support multiple with the
> > > > project - provided admin tool. It's the same reason dtests are separate
> > > -
> > > > they work with multiple versions.
> > > >
> > > > The number of repos does not affect distribution - if we want to ship
> > > > Cassandra with the admin / repair tool (we should, imo), that can be
> > > part
> > > > of the build process.
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <be...@apple.com>
> > > > wrote:
> > > >
> > > > > If the sidecar is going to be on a different release cadence, or
> > > support
> > > > > interacting with mixed mode clusters, then it should definitely be in
> > > a
> > > > > separate repo. I don’t even know how branching and merging would work
> > > in a
> > > > > repo that supports 2 separate release targets and/or mixed mode
> > > > > compatibility, but I’m pretty sure it would be a mess.
> > > > >
> > > > > As a cluster management tool, mixed mode is probably going to be a goal
> > > at
> > > > > some point. As a new project, it will benefit from not being tied to
> > > the C*
> > > > > release cycle (which would probably delay any sidecar release until
> > > > > whenever 4.1 is cut).
> > > > >
> > > > >
> > > > > On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.lynch@gmail.com)
> > >
> > > > > wrote:
> > > > >
> > > > > I think that the pros of incubating the sidecar in tree as a tool
> > > first
> > > > > outweigh the alternatives at this point of time. Rough tradeoffs that
> > > I
> > > > > see:
> > > > >
> > > > > Unique pros of in tree sidecar:
> > > > > * Faster iteration speed in general. For example when we need to add a
> > > > > new
> > > > > JMX endpoint that the sidecar needs, or change something from JMX to a
> > > > > virtual table (e.g. for repair, or monitoring) we can do all changes
> > > > > including tests as one commit within the main repository and don't
> > > have
> > > > > to
> > > > > commit to main repo, sidecar repo, and dtest repo (juggling version
> > > > > compatibility along the way).
> > > > > * We can in the future more easily move serious background
> > > functionality
> > > > > like compaction or repair itself (not repair scheduling, actual
> > > > > repairing)
> > > > > into the sidecar with a single atomic commit, we don't have to do two
> > > > > phase
> > > > > commits where we add some IPC mechanism to allow us to support it in
> > > > > both,
> > > > > then turn it on in the sidecar, then turn it off in the server, etc...
> > > > > * I think that the verification is much easier (sounds like Jonathan
> > > > > disagreed on the other thread, I could certainly be wrong), and we
> > > don't
> > > > > have to worry about testing matrices to assure that the sidecar works
> > > > > with
> > > > > various versions as the version of the sidecar that is released with
> > > that
> > > > > version of Cassandra is the only one we have to certify works. If
> > > people
> > > > > want to pull in new versions or maintain backports they can do that at
> > > > > their discretion/testing.
> > > > > * We can iterate and prove value before committing to a choice. Since
> > > it
> > > > > will be a separate artifact from the start we can always move the
> > > > > artifact
> > > > > to a separate repo later (but moving the other way is harder).
> > > > > * Users will get the sidecar "for free" when they install the daemon,
> > > > > they
> > > > > don't need to take affirmative action to e.g. be able to restart their
> > > > > cluster, run repair, or back their data up; it just comes out of the
> > > box
> > > > > for free.
> > > > >
> > > > > Unique pros of a separate repository sidecar:
> > > > > * We can use a more modern build system like gradle instead of ant
> > > > > * Merging changes is less "scary" I guess (I feel like if you're not
> > > > > touching the daemon this is already true but I could see this being
> > > less
> > > > > worrisome for some).
> > > > > * Releasing a separate artifact is somewhat easier from a separate
> > > repo
> > > > > (especially if we have gradle which makes e.g. building debs and rpms
> > > > > trivial).
> > > > > * We could backport to previous versions without getting into
> > > arguments
> > > > > about bug fixes vs features.
> > > > > * Committers could be different from the main repo, which ... may be a
> > > > > useful thing
> > > > >
> > > > > Non unique pros of a sidecar (could be achieved in the main repo or in
> > > a
> > > > > separate repo):
> > > > > * A separate build artifact .jar/.deb/.rpm that can be installed
> > > > > separately. It's slightly easier with a separate repo but certainly
> > > not
> > > > > out
> > > > > of reach within a single repo (indeed the current patch already creates
> > > a
> > > > > separate jar, and we could create a separate .deb reasonably easily).
> > > > > Personally I think having a separate .deb/.rpm is premature at this
> > > point
> > > > > (for companies that really want it they can build their own packages
> > > > > using
> > > > > the .jars), but I think it really is a distracting issue from where
> > > the
> > > > > patch should go as we can always choose to remove experimental .jar
> > > files
> > > > > that the main daemon doesn't touch.
> > > > > * A separate process lifecycle. No matter where the sidecar goes, we
> > > get
> > > > > the benefit of restarting it being less dangerous for availability
> > > than
> > > > > restarting the main daemon.
> > > > >
> > > > > That all being said, these are strong opinions weakly held and I would
> > > > > rather get something actually committed so that we can prove value one
> > > > > way
> > > > > or the other and am therefore, of course, happy to put sidecar patches
> > > > > wherever someone can review and commit it.
> > > > >
> > > > > -Joey
> > > > >
> > > > > On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <ko...@gmail.com>
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > > I am starting a new thread to get consensus on where the side car
> > > > > > should be contributed.
> > > > > >
> > > > > > Please send your responses with pro/cons of each approach or any
> > > other
> > > > > > approach. Please be clear which approach you will pick while still
> > > > > giving
> > > > > > pros/cons of both approaches.
> > > > > >
> > > > > > Thanks.
> > > > > > Sankalp
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Jon Haddad
> > > > http://www.rustyrazorblade.com
> > > > twitter: rustyrazorblade
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > >
> > >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>

Re: Side Car New Repo vs not

[Stefan Podkowinski <sp...@apache.org>]

I'm also currently -1 on the in-tree option.

Additionally to what Aleksey mentioned, I also don't see how we could
make this work with the current build and release process. Our scripts
[0] for creating releases (tarballs and native packages), would need
significant work to add support for an independent side-car. Our ant
based build process is also not a great start for adding new tasks, let
alone integrating other tool chains for web components for a potential UI.

[0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git


On 21.08.18 19:20, Aleksey Yeshchenko wrote:
> Sure, allow me to elaborate - at least a little bit. But before I do, just let me note that this wasn’t a veto -1, just a shorthand for “I don’t like this option”.
>
> It would be nice to have sidecar and C* version and release cycles fully decoupled. I know it *can* be done when in-tree, but the way we vote on releases with tags off current branches would have to change somehow. Probably painfully. It would be nice to be able to easily enforce freezes, like the upcoming one, on the whole C* repo, while allowing feature development on the sidecar. It would be nice to not have sidecar commits in emails from commits@ mailing list. It would be nice to not have C* CI trigger necessarily on sidecar commits. Groups of people working on the two repos will mostly be different too, so what’s the point in sharing the repo?
>
> Having an extra repo with its own set of branches is cheap and easy - we already do that with dtests. I like cleanly separated things when coupling is avoidable. As such I would prefer the sidecar to live in a separate new repo, while still being part of the C* project.
>
> —
> AY
>
> On 21 August 2018 at 17:06:39, sankalp kohli (kohlisankalp@gmail.com) wrote:
>
> Hi Aleksey,  
> Can you please elaborate on the reasons for your -1? This  
> way we can make progress towards any one approach.  
> Thanks,  
> Sankalp  
>
> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <al...@apple.com>  
> wrote:  
>
>> FWIW I’m strongly -1 on in-tree approach, and would much prefer a separate  
>> repo, dtest-style.  
>>  
>> —  
>> AY  
>>  
>> On 21 August 2018 at 16:36:02, Jeremiah D Jordan (  
>> jeremiah.jordan@gmail.com) wrote:  
>>  
>> I think the following is a very big plus of it being in tree:  
>>>> * Faster iteration speed in general. For example when we need to add a  
>>>> new  
>>>> JMX endpoint that the sidecar needs, or change something from JMX to a  
>>>> virtual table (e.g. for repair, or monitoring) we can do all changes  
>>>> including tests as one commit within the main repository and don't  
>> have  
>>>> to  
>>>> commit to main repo, sidecar repo,  
>>  
>> I also don’t see a reason why the sidecar being in tree means it would not  
>> work in a mixed version cluster. The nodes themselves must work in a mixed  
>> version cluster during a rolling upgrade, I would expect any management  
>> side car to operate in the same manor, in tree or not.  
>>  
>> This tool will be pretty tightly coupled with the server, and as someone  
>> with experience developing such tightly coupled tools, it is *much* easier  
>> to make sure you don’t accidentally break them if they are in tree. How  
>> many times has someone updated some JMX interface, updated nodetool, and  
>> then moved on? Breaking all the external tools not in tree, without  
>> realizing it. The above point about being able to modify interfaces and the  
>> side car in the same commit is huge in terms of making sure someone doesn’t  
>> inadvertently break the side car while fixing something else.  
>>  
>> -Jeremiah  
>>  
>>  
>>> On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>  
>> wrote:  
>>>  
>>> Strongly agree with Blake. In my mind supporting multiple versions is  
>>> mandatory. As I've stated before, we already do it with Reaper, I'd  
>>> consider it a major misstep if we couldn't support multiple with the  
>>> project - provided admin tool. It's the same reason dtests are separate  
>> -  
>>> they work with multiple versions.  
>>>  
>>> The number of repos does not affect distribution - if we want to ship  
>>> Cassandra with the admin / repair tool (we should, imo), that can be  
>> part  
>>> of the build process.  
>>>  
>>>  
>>>  
>>>  
>>> On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <be...@apple.com>  
>>> wrote:  
>>>  
>>>> If the sidecar is going to be on a different release cadence, or  
>> support  
>>>> interacting with mixed mode clusters, then it should definitely be in  
>> a  
>>>> separate repo. I don’t even know how branching and merging would work  
>> in a  
>>>> repo that supports 2 separate release targets and/or mixed mode  
>>>> compatibility, but I’m pretty sure it would be a mess.  
>>>>  
>>>> As a cluster management tool, mixed mode is probably going to be a goal  
>> at  
>>>> some point. As a new project, it will benefit from not being tied to  
>> the C*  
>>>> release cycle (which would probably delay any sidecar release until  
>>>> whenever 4.1 is cut).  
>>>>  
>>>>  
>>>> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.lynch@gmail.com)  
>>  
>>>> wrote:  
>>>>  
>>>> I think that the pros of incubating the sidecar in tree as a tool  
>> first  
>>>> outweigh the alternatives at this point of time. Rough tradeoffs that  
>> I  
>>>> see:  
>>>>  
>>>> Unique pros of in tree sidecar:  
>>>> * Faster iteration speed in general. For example when we need to add a  
>>>> new  
>>>> JMX endpoint that the sidecar needs, or change something from JMX to a  
>>>> virtual table (e.g. for repair, or monitoring) we can do all changes  
>>>> including tests as one commit within the main repository and don't  
>> have  
>>>> to  
>>>> commit to main repo, sidecar repo, and dtest repo (juggling version  
>>>> compatibility along the way).  
>>>> * We can in the future more easily move serious background  
>> functionality  
>>>> like compaction or repair itself (not repair scheduling, actual  
>>>> repairing)  
>>>> into the sidecar with a single atomic commit, we don't have to do two  
>>>> phase  
>>>> commits where we add some IPC mechanism to allow us to support it in  
>>>> both,  
>>>> then turn it on in the sidecar, then turn it off in the server, etc...  
>>>> * I think that the verification is much easier (sounds like Jonathan  
>>>> disagreed on the other thread, I could certainly be wrong), and we  
>> don't  
>>>> have to worry about testing matrices to assure that the sidecar works  
>>>> with  
>>>> various versions as the version of the sidecar that is released with  
>> that  
>>>> version of Cassandra is the only one we have to certify works. If  
>> people  
>>>> want to pull in new versions or maintain backports they can do that at  
>>>> their discretion/testing.  
>>>> * We can iterate and prove value before committing to a choice. Since  
>> it  
>>>> will be a separate artifact from the start we can always move the  
>>>> artifact  
>>>> to a separate repo later (but moving the other way is harder).  
>>>> * Users will get the sidecar "for free" when they install the daemon,  
>>>> they  
>>>> don't need to take affirmative action to e.g. be able to restart their  
>>>> cluster, run repair, or back their data up; it just comes out of the  
>> box  
>>>> for free.  
>>>>  
>>>> Unique pros of a separate repository sidecar:  
>>>> * We can use a more modern build system like gradle instead of ant  
>>>> * Merging changes is less "scary" I guess (I feel like if you're not  
>>>> touching the daemon this is already true but I could see this being  
>> less  
>>>> worrisome for some).  
>>>> * Releasing a separate artifact is somewhat easier from a separate  
>> repo  
>>>> (especially if we have gradle which makes e.g. building debs and rpms  
>>>> trivial).  
>>>> * We could backport to previous versions without getting into  
>> arguments  
>>>> about bug fixes vs features.  
>>>> * Committers could be different from the main repo, which ... may be a  
>>>> useful thing  
>>>>  
>>>> Non unique pros of a sidecar (could be achieved in the main repo or in  
>> a  
>>>> separate repo):  
>>>> * A separate build artifact .jar/.deb/.rpm that can be installed  
>>>> separately. It's slightly easier with a separate repo but certainly  
>> not  
>>>> out  
>>>> of reach within a single repo (indeed the current patch already creates  
>> a  
>>>> separate jar, and we could create a separate .deb reasonably easily).  
>>>> Personally I think having a separate .deb/.rpm is premature at this  
>> point  
>>>> (for companies that really want it they can build their own packages  
>>>> using  
>>>> the .jars), but I think it really is a distracting issue from where  
>> the  
>>>> patch should go as we can always choose to remove experimental .jar  
>> files  
>>>> that the main daemon doesn't touch.  
>>>> * A separate process lifecycle. No matter where the sidecar goes, we  
>> get  
>>>> the benefit of restarting it being less dangerous for availability  
>> than  
>>>> restarting the main daemon.  
>>>>  
>>>> That all being said, these are strong opinions weakly held and I would  
>>>> rather get something actually committed so that we can prove value one  
>>>> way  
>>>> or the other and am therefore, of course, happy to put sidecar patches  
>>>> wherever someone can review and commit it.  
>>>>  
>>>> -Joey  
>>>>  
>>>> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <ko...@gmail.com>  
>>  
>>>> wrote:  
>>>>  
>>>>> Hi,  
>>>>> I am starting a new thread to get consensus on where the side car  
>>>>> should be contributed.  
>>>>>  
>>>>> Please send your responses with pro/cons of each approach or any  
>> other  
>>>>> approach. Please be clear which approach you will pick while still  
>>>> giving  
>>>>> pros/cons of both approaches.  
>>>>>  
>>>>> Thanks.  
>>>>> Sankalp  
>>>>>  
>>>>  
>>>  
>>>  
>>> --  
>>> Jon Haddad  
>>> http://www.rustyrazorblade.com  
>>> twitter: rustyrazorblade  
>>  
>>  
>> ---------------------------------------------------------------------  
>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org  
>> For additional commands, e-mail: dev-help@cassandra.apache.org  
>>  
>>  


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org

Re: Side Car New Repo vs not

[Aleksey Yeshchenko <al...@apple.com>]

Sure, allow me to elaborate - at least a little bit. But before I do, just let me note that this wasn’t a veto -1, just a shorthand for “I don’t like this option”.

It would be nice to have sidecar and C* version and release cycles fully decoupled. I know it *can* be done when in-tree, but the way we vote on releases with tags off current branches would have to change somehow. Probably painfully. It would be nice to be able to easily enforce freezes, like the upcoming one, on the whole C* repo, while allowing feature development on the sidecar. It would be nice to not have sidecar commits in emails from commits@ mailing list. It would be nice to not have C* CI trigger necessarily on sidecar commits. Groups of people working on the two repos will mostly be different too, so what’s the point in sharing the repo?

Having an extra repo with its own set of branches is cheap and easy - we already do that with dtests. I like cleanly separated things when coupling is avoidable. As such I would prefer the sidecar to live in a separate new repo, while still being part of the C* project.

—
AY

On 21 August 2018 at 17:06:39, sankalp kohli (kohlisankalp@gmail.com) wrote:

Hi Aleksey,  
Can you please elaborate on the reasons for your -1? This  
way we can make progress towards any one approach.  
Thanks,  
Sankalp  

On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <al...@apple.com>  
wrote:  

> FWIW I’m strongly -1 on in-tree approach, and would much prefer a separate  
> repo, dtest-style.  
>  
> —  
> AY  
>  
> On 21 August 2018 at 16:36:02, Jeremiah D Jordan (  
> jeremiah.jordan@gmail.com) wrote:  
>  
> I think the following is a very big plus of it being in tree:  
> >> * Faster iteration speed in general. For example when we need to add a  
> >> new  
> >> JMX endpoint that the sidecar needs, or change something from JMX to a  
> >> virtual table (e.g. for repair, or monitoring) we can do all changes  
> >> including tests as one commit within the main repository and don't  
> have  
> >> to  
> >> commit to main repo, sidecar repo,  
>  
> I also don’t see a reason why the sidecar being in tree means it would not  
> work in a mixed version cluster. The nodes themselves must work in a mixed  
> version cluster during a rolling upgrade, I would expect any management  
> side car to operate in the same manor, in tree or not.  
>  
> This tool will be pretty tightly coupled with the server, and as someone  
> with experience developing such tightly coupled tools, it is *much* easier  
> to make sure you don’t accidentally break them if they are in tree. How  
> many times has someone updated some JMX interface, updated nodetool, and  
> then moved on? Breaking all the external tools not in tree, without  
> realizing it. The above point about being able to modify interfaces and the  
> side car in the same commit is huge in terms of making sure someone doesn’t  
> inadvertently break the side car while fixing something else.  
>  
> -Jeremiah  
>  
>  
> > On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>  
> wrote:  
> >  
> > Strongly agree with Blake. In my mind supporting multiple versions is  
> > mandatory. As I've stated before, we already do it with Reaper, I'd  
> > consider it a major misstep if we couldn't support multiple with the  
> > project - provided admin tool. It's the same reason dtests are separate  
> -  
> > they work with multiple versions.  
> >  
> > The number of repos does not affect distribution - if we want to ship  
> > Cassandra with the admin / repair tool (we should, imo), that can be  
> part  
> > of the build process.  
> >  
> >  
> >  
> >  
> > On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <be...@apple.com>  
> > wrote:  
> >  
> >> If the sidecar is going to be on a different release cadence, or  
> support  
> >> interacting with mixed mode clusters, then it should definitely be in  
> a  
> >> separate repo. I don’t even know how branching and merging would work  
> in a  
> >> repo that supports 2 separate release targets and/or mixed mode  
> >> compatibility, but I’m pretty sure it would be a mess.  
> >>  
> >> As a cluster management tool, mixed mode is probably going to be a goal  
> at  
> >> some point. As a new project, it will benefit from not being tied to  
> the C*  
> >> release cycle (which would probably delay any sidecar release until  
> >> whenever 4.1 is cut).  
> >>  
> >>  
> >> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.lynch@gmail.com)  
>  
> >> wrote:  
> >>  
> >> I think that the pros of incubating the sidecar in tree as a tool  
> first  
> >> outweigh the alternatives at this point of time. Rough tradeoffs that  
> I  
> >> see:  
> >>  
> >> Unique pros of in tree sidecar:  
> >> * Faster iteration speed in general. For example when we need to add a  
> >> new  
> >> JMX endpoint that the sidecar needs, or change something from JMX to a  
> >> virtual table (e.g. for repair, or monitoring) we can do all changes  
> >> including tests as one commit within the main repository and don't  
> have  
> >> to  
> >> commit to main repo, sidecar repo, and dtest repo (juggling version  
> >> compatibility along the way).  
> >> * We can in the future more easily move serious background  
> functionality  
> >> like compaction or repair itself (not repair scheduling, actual  
> >> repairing)  
> >> into the sidecar with a single atomic commit, we don't have to do two  
> >> phase  
> >> commits where we add some IPC mechanism to allow us to support it in  
> >> both,  
> >> then turn it on in the sidecar, then turn it off in the server, etc...  
> >> * I think that the verification is much easier (sounds like Jonathan  
> >> disagreed on the other thread, I could certainly be wrong), and we  
> don't  
> >> have to worry about testing matrices to assure that the sidecar works  
> >> with  
> >> various versions as the version of the sidecar that is released with  
> that  
> >> version of Cassandra is the only one we have to certify works. If  
> people  
> >> want to pull in new versions or maintain backports they can do that at  
> >> their discretion/testing.  
> >> * We can iterate and prove value before committing to a choice. Since  
> it  
> >> will be a separate artifact from the start we can always move the  
> >> artifact  
> >> to a separate repo later (but moving the other way is harder).  
> >> * Users will get the sidecar "for free" when they install the daemon,  
> >> they  
> >> don't need to take affirmative action to e.g. be able to restart their  
> >> cluster, run repair, or back their data up; it just comes out of the  
> box  
> >> for free.  
> >>  
> >> Unique pros of a separate repository sidecar:  
> >> * We can use a more modern build system like gradle instead of ant  
> >> * Merging changes is less "scary" I guess (I feel like if you're not  
> >> touching the daemon this is already true but I could see this being  
> less  
> >> worrisome for some).  
> >> * Releasing a separate artifact is somewhat easier from a separate  
> repo  
> >> (especially if we have gradle which makes e.g. building debs and rpms  
> >> trivial).  
> >> * We could backport to previous versions without getting into  
> arguments  
> >> about bug fixes vs features.  
> >> * Committers could be different from the main repo, which ... may be a  
> >> useful thing  
> >>  
> >> Non unique pros of a sidecar (could be achieved in the main repo or in  
> a  
> >> separate repo):  
> >> * A separate build artifact .jar/.deb/.rpm that can be installed  
> >> separately. It's slightly easier with a separate repo but certainly  
> not  
> >> out  
> >> of reach within a single repo (indeed the current patch already creates  
> a  
> >> separate jar, and we could create a separate .deb reasonably easily).  
> >> Personally I think having a separate .deb/.rpm is premature at this  
> point  
> >> (for companies that really want it they can build their own packages  
> >> using  
> >> the .jars), but I think it really is a distracting issue from where  
> the  
> >> patch should go as we can always choose to remove experimental .jar  
> files  
> >> that the main daemon doesn't touch.  
> >> * A separate process lifecycle. No matter where the sidecar goes, we  
> get  
> >> the benefit of restarting it being less dangerous for availability  
> than  
> >> restarting the main daemon.  
> >>  
> >> That all being said, these are strong opinions weakly held and I would  
> >> rather get something actually committed so that we can prove value one  
> >> way  
> >> or the other and am therefore, of course, happy to put sidecar patches  
> >> wherever someone can review and commit it.  
> >>  
> >> -Joey  
> >>  
> >> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <ko...@gmail.com>  
>  
> >> wrote:  
> >>  
> >>> Hi,  
> >>> I am starting a new thread to get consensus on where the side car  
> >>> should be contributed.  
> >>>  
> >>> Please send your responses with pro/cons of each approach or any  
> other  
> >>> approach. Please be clear which approach you will pick while still  
> >> giving  
> >>> pros/cons of both approaches.  
> >>>  
> >>> Thanks.  
> >>> Sankalp  
> >>>  
> >>  
> >  
> >  
> > --  
> > Jon Haddad  
> > http://www.rustyrazorblade.com  
> > twitter: rustyrazorblade  
>  
>  
> ---------------------------------------------------------------------  
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org  
> For additional commands, e-mail: dev-help@cassandra.apache.org  
>  
>  

Re: Side Car New Repo vs not

[sankalp kohli <ko...@gmail.com>]

Hi Aleksey,
                 Can you please elaborate on the reasons for your -1? This
way we can make progress towards any one approach.
Thanks,
Sankalp

On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <al...@apple.com>
wrote:

> FWIW I’m strongly -1 on in-tree approach, and would much prefer a separate
> repo, dtest-style.
>
> —
> AY
>
> On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> jeremiah.jordan@gmail.com) wrote:
>
> I think the following is a very big plus of it being in tree:
> >> * Faster iteration speed in general. For example when we need to add a
> >> new
> >> JMX endpoint that the sidecar needs, or change something from JMX to a
> >> virtual table (e.g. for repair, or monitoring) we can do all changes
> >> including tests as one commit within the main repository and don't
> have
> >> to
> >> commit to main repo, sidecar repo,
>
> I also don’t see a reason why the sidecar being in tree means it would not
> work in a mixed version cluster. The nodes themselves must work in a mixed
> version cluster during a rolling upgrade, I would expect any management
> side car to operate in the same manor, in tree or not.
>
> This tool will be pretty tightly coupled with the server, and as someone
> with experience developing such tightly coupled tools, it is *much* easier
> to make sure you don’t accidentally break them if they are in tree. How
> many times has someone updated some JMX interface, updated nodetool, and
> then moved on? Breaking all the external tools not in tree, without
> realizing it. The above point about being able to modify interfaces and the
> side car in the same commit is huge in terms of making sure someone doesn’t
> inadvertently break the side car while fixing something else.
>
> -Jeremiah
>
>
> > On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com>
> wrote:
> >
> > Strongly agree with Blake. In my mind supporting multiple versions is
> > mandatory. As I've stated before, we already do it with Reaper, I'd
> > consider it a major misstep if we couldn't support multiple with the
> > project - provided admin tool. It's the same reason dtests are separate
> -
> > they work with multiple versions.
> >
> > The number of repos does not affect distribution - if we want to ship
> > Cassandra with the admin / repair tool (we should, imo), that can be
> part
> > of the build process.
> >
> >
> >
> >
> > On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <be...@apple.com>
> > wrote:
> >
> >> If the sidecar is going to be on a different release cadence, or
> support
> >> interacting with mixed mode clusters, then it should definitely be in
> a
> >> separate repo. I don’t even know how branching and merging would work
> in a
> >> repo that supports 2 separate release targets and/or mixed mode
> >> compatibility, but I’m pretty sure it would be a mess.
> >>
> >> As a cluster management tool, mixed mode is probably going to be a goal
> at
> >> some point. As a new project, it will benefit from not being tied to
> the C*
> >> release cycle (which would probably delay any sidecar release until
> >> whenever 4.1 is cut).
> >>
> >>
> >> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.lynch@gmail.com)
>
> >> wrote:
> >>
> >> I think that the pros of incubating the sidecar in tree as a tool
> first
> >> outweigh the alternatives at this point of time. Rough tradeoffs that
> I
> >> see:
> >>
> >> Unique pros of in tree sidecar:
> >> * Faster iteration speed in general. For example when we need to add a
> >> new
> >> JMX endpoint that the sidecar needs, or change something from JMX to a
> >> virtual table (e.g. for repair, or monitoring) we can do all changes
> >> including tests as one commit within the main repository and don't
> have
> >> to
> >> commit to main repo, sidecar repo, and dtest repo (juggling version
> >> compatibility along the way).
> >> * We can in the future more easily move serious background
> functionality
> >> like compaction or repair itself (not repair scheduling, actual
> >> repairing)
> >> into the sidecar with a single atomic commit, we don't have to do two
> >> phase
> >> commits where we add some IPC mechanism to allow us to support it in
> >> both,
> >> then turn it on in the sidecar, then turn it off in the server, etc...
> >> * I think that the verification is much easier (sounds like Jonathan
> >> disagreed on the other thread, I could certainly be wrong), and we
> don't
> >> have to worry about testing matrices to assure that the sidecar works
> >> with
> >> various versions as the version of the sidecar that is released with
> that
> >> version of Cassandra is the only one we have to certify works. If
> people
> >> want to pull in new versions or maintain backports they can do that at
> >> their discretion/testing.
> >> * We can iterate and prove value before committing to a choice. Since
> it
> >> will be a separate artifact from the start we can always move the
> >> artifact
> >> to a separate repo later (but moving the other way is harder).
> >> * Users will get the sidecar "for free" when they install the daemon,
> >> they
> >> don't need to take affirmative action to e.g. be able to restart their
> >> cluster, run repair, or back their data up; it just comes out of the
> box
> >> for free.
> >>
> >> Unique pros of a separate repository sidecar:
> >> * We can use a more modern build system like gradle instead of ant
> >> * Merging changes is less "scary" I guess (I feel like if you're not
> >> touching the daemon this is already true but I could see this being
> less
> >> worrisome for some).
> >> * Releasing a separate artifact is somewhat easier from a separate
> repo
> >> (especially if we have gradle which makes e.g. building debs and rpms
> >> trivial).
> >> * We could backport to previous versions without getting into
> arguments
> >> about bug fixes vs features.
> >> * Committers could be different from the main repo, which ... may be a
> >> useful thing
> >>
> >> Non unique pros of a sidecar (could be achieved in the main repo or in
> a
> >> separate repo):
> >> * A separate build artifact .jar/.deb/.rpm that can be installed
> >> separately. It's slightly easier with a separate repo but certainly
> not
> >> out
> >> of reach within a single repo (indeed the current patch already creates
> a
> >> separate jar, and we could create a separate .deb reasonably easily).
> >> Personally I think having a separate .deb/.rpm is premature at this
> point
> >> (for companies that really want it they can build their own packages
> >> using
> >> the .jars), but I think it really is a distracting issue from where
> the
> >> patch should go as we can always choose to remove experimental .jar
> files
> >> that the main daemon doesn't touch.
> >> * A separate process lifecycle. No matter where the sidecar goes, we
> get
> >> the benefit of restarting it being less dangerous for availability
> than
> >> restarting the main daemon.
> >>
> >> That all being said, these are strong opinions weakly held and I would
> >> rather get something actually committed so that we can prove value one
> >> way
> >> or the other and am therefore, of course, happy to put sidecar patches
> >> wherever someone can review and commit it.
> >>
> >> -Joey
> >>
> >> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <ko...@gmail.com>
>
> >> wrote:
> >>
> >>> Hi,
> >>> I am starting a new thread to get consensus on where the side car
> >>> should be contributed.
> >>>
> >>> Please send your responses with pro/cons of each approach or any
> other
> >>> approach. Please be clear which approach you will pick while still
> >> giving
> >>> pros/cons of both approaches.
> >>>
> >>> Thanks.
> >>> Sankalp
> >>>
> >>
> >
> >
> > --
> > Jon Haddad
> > http://www.rustyrazorblade.com
> > twitter: rustyrazorblade
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>

Re: Side Car New Repo vs not

[Aleksey Yeshchenko <al...@apple.com>]

FWIW I’m strongly -1 on in-tree approach, and would much prefer a separate repo, dtest-style.

—
AY

On 21 August 2018 at 16:36:02, Jeremiah D Jordan (jeremiah.jordan@gmail.com) wrote:

I think the following is a very big plus of it being in tree:  
>> * Faster iteration speed in general. For example when we need to add a  
>> new  
>> JMX endpoint that the sidecar needs, or change something from JMX to a  
>> virtual table (e.g. for repair, or monitoring) we can do all changes  
>> including tests as one commit within the main repository and don't have  
>> to  
>> commit to main repo, sidecar repo,  

I also don’t see a reason why the sidecar being in tree means it would not work in a mixed version cluster. The nodes themselves must work in a mixed version cluster during a rolling upgrade, I would expect any management side car to operate in the same manor, in tree or not.  

This tool will be pretty tightly coupled with the server, and as someone with experience developing such tightly coupled tools, it is *much* easier to make sure you don’t accidentally break them if they are in tree. How many times has someone updated some JMX interface, updated nodetool, and then moved on? Breaking all the external tools not in tree, without realizing it. The above point about being able to modify interfaces and the side car in the same commit is huge in terms of making sure someone doesn’t inadvertently break the side car while fixing something else.  

-Jeremiah  


> On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com> wrote:  
>  
> Strongly agree with Blake. In my mind supporting multiple versions is  
> mandatory. As I've stated before, we already do it with Reaper, I'd  
> consider it a major misstep if we couldn't support multiple with the  
> project - provided admin tool. It's the same reason dtests are separate -  
> they work with multiple versions.  
>  
> The number of repos does not affect distribution - if we want to ship  
> Cassandra with the admin / repair tool (we should, imo), that can be part  
> of the build process.  
>  
>  
>  
>  
> On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <be...@apple.com>  
> wrote:  
>  
>> If the sidecar is going to be on a different release cadence, or support  
>> interacting with mixed mode clusters, then it should definitely be in a  
>> separate repo. I don’t even know how branching and merging would work in a  
>> repo that supports 2 separate release targets and/or mixed mode  
>> compatibility, but I’m pretty sure it would be a mess.  
>>  
>> As a cluster management tool, mixed mode is probably going to be a goal at  
>> some point. As a new project, it will benefit from not being tied to the C*  
>> release cycle (which would probably delay any sidecar release until  
>> whenever 4.1 is cut).  
>>  
>>  
>> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.lynch@gmail.com)  
>> wrote:  
>>  
>> I think that the pros of incubating the sidecar in tree as a tool first  
>> outweigh the alternatives at this point of time. Rough tradeoffs that I  
>> see:  
>>  
>> Unique pros of in tree sidecar:  
>> * Faster iteration speed in general. For example when we need to add a  
>> new  
>> JMX endpoint that the sidecar needs, or change something from JMX to a  
>> virtual table (e.g. for repair, or monitoring) we can do all changes  
>> including tests as one commit within the main repository and don't have  
>> to  
>> commit to main repo, sidecar repo, and dtest repo (juggling version  
>> compatibility along the way).  
>> * We can in the future more easily move serious background functionality  
>> like compaction or repair itself (not repair scheduling, actual  
>> repairing)  
>> into the sidecar with a single atomic commit, we don't have to do two  
>> phase  
>> commits where we add some IPC mechanism to allow us to support it in  
>> both,  
>> then turn it on in the sidecar, then turn it off in the server, etc...  
>> * I think that the verification is much easier (sounds like Jonathan  
>> disagreed on the other thread, I could certainly be wrong), and we don't  
>> have to worry about testing matrices to assure that the sidecar works  
>> with  
>> various versions as the version of the sidecar that is released with that  
>> version of Cassandra is the only one we have to certify works. If people  
>> want to pull in new versions or maintain backports they can do that at  
>> their discretion/testing.  
>> * We can iterate and prove value before committing to a choice. Since it  
>> will be a separate artifact from the start we can always move the  
>> artifact  
>> to a separate repo later (but moving the other way is harder).  
>> * Users will get the sidecar "for free" when they install the daemon,  
>> they  
>> don't need to take affirmative action to e.g. be able to restart their  
>> cluster, run repair, or back their data up; it just comes out of the box  
>> for free.  
>>  
>> Unique pros of a separate repository sidecar:  
>> * We can use a more modern build system like gradle instead of ant  
>> * Merging changes is less "scary" I guess (I feel like if you're not  
>> touching the daemon this is already true but I could see this being less  
>> worrisome for some).  
>> * Releasing a separate artifact is somewhat easier from a separate repo  
>> (especially if we have gradle which makes e.g. building debs and rpms  
>> trivial).  
>> * We could backport to previous versions without getting into arguments  
>> about bug fixes vs features.  
>> * Committers could be different from the main repo, which ... may be a  
>> useful thing  
>>  
>> Non unique pros of a sidecar (could be achieved in the main repo or in a  
>> separate repo):  
>> * A separate build artifact .jar/.deb/.rpm that can be installed  
>> separately. It's slightly easier with a separate repo but certainly not  
>> out  
>> of reach within a single repo (indeed the current patch already creates a  
>> separate jar, and we could create a separate .deb reasonably easily).  
>> Personally I think having a separate .deb/.rpm is premature at this point  
>> (for companies that really want it they can build their own packages  
>> using  
>> the .jars), but I think it really is a distracting issue from where the  
>> patch should go as we can always choose to remove experimental .jar files  
>> that the main daemon doesn't touch.  
>> * A separate process lifecycle. No matter where the sidecar goes, we get  
>> the benefit of restarting it being less dangerous for availability than  
>> restarting the main daemon.  
>>  
>> That all being said, these are strong opinions weakly held and I would  
>> rather get something actually committed so that we can prove value one  
>> way  
>> or the other and am therefore, of course, happy to put sidecar patches  
>> wherever someone can review and commit it.  
>>  
>> -Joey  
>>  
>> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <ko...@gmail.com>  
>> wrote:  
>>  
>>> Hi,  
>>> I am starting a new thread to get consensus on where the side car  
>>> should be contributed.  
>>>  
>>> Please send your responses with pro/cons of each approach or any other  
>>> approach. Please be clear which approach you will pick while still  
>> giving  
>>> pros/cons of both approaches.  
>>>  
>>> Thanks.  
>>> Sankalp  
>>>  
>>  
>  
>  
> --  
> Jon Haddad  
> http://www.rustyrazorblade.com  
> twitter: rustyrazorblade  


---------------------------------------------------------------------  
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org  
For additional commands, e-mail: dev-help@cassandra.apache.org  

Re: Side Car New Repo vs not

[Jeremiah D Jordan <je...@gmail.com>]

I think the following is a very big plus of it being in tree:
>> * Faster iteration speed in general. For example when we need to add a
>> new
>> JMX endpoint that the sidecar needs, or change something from JMX to a
>> virtual table (e.g. for repair, or monitoring) we can do all changes
>> including tests as one commit within the main repository and don't have
>> to
>> commit to main repo, sidecar repo, 

I also don’t see a reason why the sidecar being in tree means it would not work in a mixed version cluster.  The nodes themselves must work in a mixed version cluster during a rolling upgrade, I would expect any management side car to operate in the same manor, in tree or not.

This tool will be pretty tightly coupled with the server, and as someone with experience developing such tightly coupled tools, it is *much* easier to make sure you don’t accidentally break them if they are in tree.  How many times has someone updated some JMX interface, updated nodetool, and then moved on?  Breaking all the external tools not in tree, without realizing it.  The above point about being able to modify interfaces and the side car in the same commit is huge in terms of making sure someone doesn’t inadvertently break the side car while fixing something else.

-Jeremiah


> On Aug 21, 2018, at 10:28 AM, Jonathan Haddad <jo...@jonhaddad.com> wrote:
> 
> Strongly agree with Blake.  In my mind supporting multiple versions is
> mandatory.  As I've stated before, we already do it with Reaper, I'd
> consider it a major misstep if we couldn't support multiple with the
> project - provided admin tool.  It's the same reason dtests are separate -
> they work with multiple versions.
> 
> The number of repos does not affect distribution - if we want to ship
> Cassandra with the admin / repair tool (we should, imo), that can be part
> of the build process.
> 
> 
> 
> 
> On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <be...@apple.com>
> wrote:
> 
>> If the sidecar is going to be on a different release cadence, or support
>> interacting with mixed mode clusters, then it should definitely be in a
>> separate repo. I don’t even know how branching and merging would work in a
>> repo that supports 2 separate release targets and/or mixed mode
>> compatibility, but I’m pretty sure it would be a mess.
>> 
>> As a cluster management tool, mixed mode is probably going to be a goal at
>> some point. As a new project, it will benefit from not being tied to the C*
>> release cycle (which would probably delay any sidecar release until
>> whenever 4.1 is cut).
>> 
>> 
>> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.lynch@gmail.com)
>> wrote:
>> 
>> I think that the pros of incubating the sidecar in tree as a tool first
>> outweigh the alternatives at this point of time. Rough tradeoffs that I
>> see:
>> 
>> Unique pros of in tree sidecar:
>> * Faster iteration speed in general. For example when we need to add a
>> new
>> JMX endpoint that the sidecar needs, or change something from JMX to a
>> virtual table (e.g. for repair, or monitoring) we can do all changes
>> including tests as one commit within the main repository and don't have
>> to
>> commit to main repo, sidecar repo, and dtest repo (juggling version
>> compatibility along the way).
>> * We can in the future more easily move serious background functionality
>> like compaction or repair itself (not repair scheduling, actual
>> repairing)
>> into the sidecar with a single atomic commit, we don't have to do two
>> phase
>> commits where we add some IPC mechanism to allow us to support it in
>> both,
>> then turn it on in the sidecar, then turn it off in the server, etc...
>> * I think that the verification is much easier (sounds like Jonathan
>> disagreed on the other thread, I could certainly be wrong), and we don't
>> have to worry about testing matrices to assure that the sidecar works
>> with
>> various versions as the version of the sidecar that is released with that
>> version of Cassandra is the only one we have to certify works. If people
>> want to pull in new versions or maintain backports they can do that at
>> their discretion/testing.
>> * We can iterate and prove value before committing to a choice. Since it
>> will be a separate artifact from the start we can always move the
>> artifact
>> to a separate repo later (but moving the other way is harder).
>> * Users will get the sidecar "for free" when they install the daemon,
>> they
>> don't need to take affirmative action to e.g. be able to restart their
>> cluster, run repair, or back their data up; it just comes out of the box
>> for free.
>> 
>> Unique pros of a separate repository sidecar:
>> * We can use a more modern build system like gradle instead of ant
>> * Merging changes is less "scary" I guess (I feel like if you're not
>> touching the daemon this is already true but I could see this being less
>> worrisome for some).
>> * Releasing a separate artifact is somewhat easier from a separate repo
>> (especially if we have gradle which makes e.g. building debs and rpms
>> trivial).
>> * We could backport to previous versions without getting into arguments
>> about bug fixes vs features.
>> * Committers could be different from the main repo, which ... may be a
>> useful thing
>> 
>> Non unique pros of a sidecar (could be achieved in the main repo or in a
>> separate repo):
>> * A separate build artifact .jar/.deb/.rpm that can be installed
>> separately. It's slightly easier with a separate repo but certainly not
>> out
>> of reach within a single repo (indeed the current patch already creates a
>> separate jar, and we could create a separate .deb reasonably easily).
>> Personally I think having a separate .deb/.rpm is premature at this point
>> (for companies that really want it they can build their own packages
>> using
>> the .jars), but I think it really is a distracting issue from where the
>> patch should go as we can always choose to remove experimental .jar files
>> that the main daemon doesn't touch.
>> * A separate process lifecycle. No matter where the sidecar goes, we get
>> the benefit of restarting it being less dangerous for availability than
>> restarting the main daemon.
>> 
>> That all being said, these are strong opinions weakly held and I would
>> rather get something actually committed so that we can prove value one
>> way
>> or the other and am therefore, of course, happy to put sidecar patches
>> wherever someone can review and commit it.
>> 
>> -Joey
>> 
>> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <ko...@gmail.com>
>> wrote:
>> 
>>> Hi,
>>> I am starting a new thread to get consensus on where the side car
>>> should be contributed.
>>> 
>>> Please send your responses with pro/cons of each approach or any other
>>> approach. Please be clear which approach you will pick while still
>> giving
>>> pros/cons of both approaches.
>>> 
>>> Thanks.
>>> Sankalp
>>> 
>> 
> 
> 
> -- 
> Jon Haddad
> http://www.rustyrazorblade.com
> twitter: rustyrazorblade


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org

Re: Side Car New Repo vs not

[Jonathan Haddad <jo...@jonhaddad.com>]

Strongly agree with Blake.  In my mind supporting multiple versions is
mandatory.  As I've stated before, we already do it with Reaper, I'd
consider it a major misstep if we couldn't support multiple with the
project - provided admin tool.  It's the same reason dtests are separate -
they work with multiple versions.

The number of repos does not affect distribution - if we want to ship
Cassandra with the admin / repair tool (we should, imo), that can be part
of the build process.




On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston <be...@apple.com>
wrote:

> If the sidecar is going to be on a different release cadence, or support
> interacting with mixed mode clusters, then it should definitely be in a
> separate repo. I don’t even know how branching and merging would work in a
> repo that supports 2 separate release targets and/or mixed mode
> compatibility, but I’m pretty sure it would be a mess.
>
> As a cluster management tool, mixed mode is probably going to be a goal at
> some point. As a new project, it will benefit from not being tied to the C*
> release cycle (which would probably delay any sidecar release until
> whenever 4.1 is cut).
>
>
> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.lynch@gmail.com)
> wrote:
>
> I think that the pros of incubating the sidecar in tree as a tool first
> outweigh the alternatives at this point of time. Rough tradeoffs that I
> see:
>
> Unique pros of in tree sidecar:
> * Faster iteration speed in general. For example when we need to add a
> new
> JMX endpoint that the sidecar needs, or change something from JMX to a
> virtual table (e.g. for repair, or monitoring) we can do all changes
> including tests as one commit within the main repository and don't have
> to
> commit to main repo, sidecar repo, and dtest repo (juggling version
> compatibility along the way).
> * We can in the future more easily move serious background functionality
> like compaction or repair itself (not repair scheduling, actual
> repairing)
> into the sidecar with a single atomic commit, we don't have to do two
> phase
> commits where we add some IPC mechanism to allow us to support it in
> both,
> then turn it on in the sidecar, then turn it off in the server, etc...
> * I think that the verification is much easier (sounds like Jonathan
> disagreed on the other thread, I could certainly be wrong), and we don't
> have to worry about testing matrices to assure that the sidecar works
> with
> various versions as the version of the sidecar that is released with that
> version of Cassandra is the only one we have to certify works. If people
> want to pull in new versions or maintain backports they can do that at
> their discretion/testing.
> * We can iterate and prove value before committing to a choice. Since it
> will be a separate artifact from the start we can always move the
> artifact
> to a separate repo later (but moving the other way is harder).
> * Users will get the sidecar "for free" when they install the daemon,
> they
> don't need to take affirmative action to e.g. be able to restart their
> cluster, run repair, or back their data up; it just comes out of the box
> for free.
>
> Unique pros of a separate repository sidecar:
> * We can use a more modern build system like gradle instead of ant
> * Merging changes is less "scary" I guess (I feel like if you're not
> touching the daemon this is already true but I could see this being less
> worrisome for some).
> * Releasing a separate artifact is somewhat easier from a separate repo
> (especially if we have gradle which makes e.g. building debs and rpms
> trivial).
> * We could backport to previous versions without getting into arguments
> about bug fixes vs features.
> * Committers could be different from the main repo, which ... may be a
> useful thing
>
> Non unique pros of a sidecar (could be achieved in the main repo or in a
> separate repo):
> * A separate build artifact .jar/.deb/.rpm that can be installed
> separately. It's slightly easier with a separate repo but certainly not
> out
> of reach within a single repo (indeed the current patch already creates a
> separate jar, and we could create a separate .deb reasonably easily).
> Personally I think having a separate .deb/.rpm is premature at this point
> (for companies that really want it they can build their own packages
> using
> the .jars), but I think it really is a distracting issue from where the
> patch should go as we can always choose to remove experimental .jar files
> that the main daemon doesn't touch.
> * A separate process lifecycle. No matter where the sidecar goes, we get
> the benefit of restarting it being less dangerous for availability than
> restarting the main daemon.
>
> That all being said, these are strong opinions weakly held and I would
> rather get something actually committed so that we can prove value one
> way
> or the other and am therefore, of course, happy to put sidecar patches
> wherever someone can review and commit it.
>
> -Joey
>
> On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <ko...@gmail.com>
> wrote:
>
> > Hi,
> > I am starting a new thread to get consensus on where the side car
> > should be contributed.
> >
> > Please send your responses with pro/cons of each approach or any other
> > approach. Please be clear which approach you will pick while still
> giving
> > pros/cons of both approaches.
> >
> > Thanks.
> > Sankalp
> >
>


-- 
Jon Haddad
http://www.rustyrazorblade.com
twitter: rustyrazorblade

Re: Side Car New Repo vs not

["dinesh.joshi@yahoo.com.INVALID" <di...@yahoo.com.INVALID>]

An option is to create a mono repo with Cassandra and SideCar as modules that could be built independently. This would keep source for both artifacts in the same repo and have their own release cadences. That said, I don't have any strong opinions at this point. We can try going with a separate repo and reevaluate it if it doesn't work out.
Dinesh 

    On Monday, August 20, 2018, 9:21:33 PM PDT, Blake Eggleston <be...@apple.com> wrote:  
 
 If the sidecar is going to be on a different release cadence, or support interacting with mixed mode clusters, then it should definitely be in a separate repo. I don’t even know how branching and merging would work in a repo that supports 2 separate release targets and/or mixed mode compatibility, but I’m pretty sure it would be a mess.

As a cluster management tool, mixed mode is probably going to be a goal at some point. As a new project, it will benefit from not being tied to the C* release cycle (which would probably delay any sidecar release until whenever 4.1 is cut).


On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.lynch@gmail.com) wrote:

I think that the pros of incubating the sidecar in tree as a tool first  
outweigh the alternatives at this point of time. Rough tradeoffs that I see:  

Unique pros of in tree sidecar:  
* Faster iteration speed in general. For example when we need to add a new  
JMX endpoint that the sidecar needs, or change something from JMX to a  
virtual table (e.g. for repair, or monitoring) we can do all changes  
including tests as one commit within the main repository and don't have to  
commit to main repo, sidecar repo, and dtest repo (juggling version  
compatibility along the way).  
* We can in the future more easily move serious background functionality  
like compaction or repair itself (not repair scheduling, actual repairing)  
into the sidecar with a single atomic commit, we don't have to do two phase  
commits where we add some IPC mechanism to allow us to support it in both,  
then turn it on in the sidecar, then turn it off in the server, etc...  
* I think that the verification is much easier (sounds like Jonathan  
disagreed on the other thread, I could certainly be wrong), and we don't  
have to worry about testing matrices to assure that the sidecar works with  
various versions as the version of the sidecar that is released with that  
version of Cassandra is the only one we have to certify works. If people  
want to pull in new versions or maintain backports they can do that at  
their discretion/testing.  
* We can iterate and prove value before committing to a choice. Since it  
will be a separate artifact from the start we can always move the artifact  
to a separate repo later (but moving the other way is harder).  
* Users will get the sidecar "for free" when they install the daemon, they  
don't need to take affirmative action to e.g. be able to restart their  
cluster, run repair, or back their data up; it just comes out of the box  
for free.  

Unique pros of a separate repository sidecar:  
* We can use a more modern build system like gradle instead of ant  
* Merging changes is less "scary" I guess (I feel like if you're not  
touching the daemon this is already true but I could see this being less  
worrisome for some).  
* Releasing a separate artifact is somewhat easier from a separate repo  
(especially if we have gradle which makes e.g. building debs and rpms  
trivial).  
* We could backport to previous versions without getting into arguments  
about bug fixes vs features.  
* Committers could be different from the main repo, which ... may be a  
useful thing  

Non unique pros of a sidecar (could be achieved in the main repo or in a  
separate repo):  
* A separate build artifact .jar/.deb/.rpm that can be installed  
separately. It's slightly easier with a separate repo but certainly not out  
of reach within a single repo (indeed the current patch already creates a  
separate jar, and we could create a separate .deb reasonably easily).  
Personally I think having a separate .deb/.rpm is premature at this point  
(for companies that really want it they can build their own packages using  
the .jars), but I think it really is a distracting issue from where the  
patch should go as we can always choose to remove experimental .jar files  
that the main daemon doesn't touch.  
* A separate process lifecycle. No matter where the sidecar goes, we get  
the benefit of restarting it being less dangerous for availability than  
restarting the main daemon.  

That all being said, these are strong opinions weakly held and I would  
rather get something actually committed so that we can prove value one way  
or the other and am therefore, of course, happy to put sidecar patches  
wherever someone can review and commit it.  

-Joey  

On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <ko...@gmail.com>  
wrote:  

> Hi,  
> I am starting a new thread to get consensus on where the side car  
> should be contributed.  
>  
> Please send your responses with pro/cons of each approach or any other  
> approach. Please be clear which approach you will pick while still giving  
> pros/cons of both approaches.  
>  
> Thanks.  
> Sankalp  
>    

Re: Side Car New Repo vs not

[Blake Eggleston <be...@apple.com>]

If the sidecar is going to be on a different release cadence, or support interacting with mixed mode clusters, then it should definitely be in a separate repo. I don’t even know how branching and merging would work in a repo that supports 2 separate release targets and/or mixed mode compatibility, but I’m pretty sure it would be a mess.

As a cluster management tool, mixed mode is probably going to be a goal at some point. As a new project, it will benefit from not being tied to the C* release cycle (which would probably delay any sidecar release until whenever 4.1 is cut).


On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.lynch@gmail.com) wrote:

I think that the pros of incubating the sidecar in tree as a tool first  
outweigh the alternatives at this point of time. Rough tradeoffs that I see:  

Unique pros of in tree sidecar:  
* Faster iteration speed in general. For example when we need to add a new  
JMX endpoint that the sidecar needs, or change something from JMX to a  
virtual table (e.g. for repair, or monitoring) we can do all changes  
including tests as one commit within the main repository and don't have to  
commit to main repo, sidecar repo, and dtest repo (juggling version  
compatibility along the way).  
* We can in the future more easily move serious background functionality  
like compaction or repair itself (not repair scheduling, actual repairing)  
into the sidecar with a single atomic commit, we don't have to do two phase  
commits where we add some IPC mechanism to allow us to support it in both,  
then turn it on in the sidecar, then turn it off in the server, etc...  
* I think that the verification is much easier (sounds like Jonathan  
disagreed on the other thread, I could certainly be wrong), and we don't  
have to worry about testing matrices to assure that the sidecar works with  
various versions as the version of the sidecar that is released with that  
version of Cassandra is the only one we have to certify works. If people  
want to pull in new versions or maintain backports they can do that at  
their discretion/testing.  
* We can iterate and prove value before committing to a choice. Since it  
will be a separate artifact from the start we can always move the artifact  
to a separate repo later (but moving the other way is harder).  
* Users will get the sidecar "for free" when they install the daemon, they  
don't need to take affirmative action to e.g. be able to restart their  
cluster, run repair, or back their data up; it just comes out of the box  
for free.  

Unique pros of a separate repository sidecar:  
* We can use a more modern build system like gradle instead of ant  
* Merging changes is less "scary" I guess (I feel like if you're not  
touching the daemon this is already true but I could see this being less  
worrisome for some).  
* Releasing a separate artifact is somewhat easier from a separate repo  
(especially if we have gradle which makes e.g. building debs and rpms  
trivial).  
* We could backport to previous versions without getting into arguments  
about bug fixes vs features.  
* Committers could be different from the main repo, which ... may be a  
useful thing  

Non unique pros of a sidecar (could be achieved in the main repo or in a  
separate repo):  
* A separate build artifact .jar/.deb/.rpm that can be installed  
separately. It's slightly easier with a separate repo but certainly not out  
of reach within a single repo (indeed the current patch already creates a  
separate jar, and we could create a separate .deb reasonably easily).  
Personally I think having a separate .deb/.rpm is premature at this point  
(for companies that really want it they can build their own packages using  
the .jars), but I think it really is a distracting issue from where the  
patch should go as we can always choose to remove experimental .jar files  
that the main daemon doesn't touch.  
* A separate process lifecycle. No matter where the sidecar goes, we get  
the benefit of restarting it being less dangerous for availability than  
restarting the main daemon.  

That all being said, these are strong opinions weakly held and I would  
rather get something actually committed so that we can prove value one way  
or the other and am therefore, of course, happy to put sidecar patches  
wherever someone can review and commit it.  

-Joey  

On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <ko...@gmail.com>  
wrote:  

> Hi,  
> I am starting a new thread to get consensus on where the side car  
> should be contributed.  
>  
> Please send your responses with pro/cons of each approach or any other  
> approach. Please be clear which approach you will pick while still giving  
> pros/cons of both approaches.  
>  
> Thanks.  
> Sankalp  
>  

Re: Side Car New Repo vs not

[Joseph Lynch <jo...@gmail.com>]

I think that the pros of incubating the sidecar in tree as a tool first
outweigh the alternatives at this point of time. Rough tradeoffs that I see:

Unique pros of in tree sidecar:
* Faster iteration speed in general. For example when we need to add a new
JMX endpoint that the sidecar needs, or change something from JMX to a
virtual table (e.g. for repair, or monitoring) we can do all changes
including tests as one commit within the main repository and don't have to
commit to main repo, sidecar repo, and dtest repo (juggling version
compatibility along the way).
* We can in the future more easily move serious background functionality
like compaction or repair itself (not repair scheduling, actual repairing)
into the sidecar with a single atomic commit, we don't have to do two phase
commits where we add some IPC mechanism to allow us to support it in both,
then turn it on in the sidecar, then turn it off in the server, etc...
* I think that the verification is much easier (sounds like Jonathan
disagreed on the other thread, I could certainly be wrong), and we don't
have to worry about testing matrices to assure that the sidecar works with
various versions as the version of the sidecar that is released with that
version of Cassandra is the only one we have to certify works. If people
want to pull in new versions or maintain backports they can do that at
their discretion/testing.
* We can iterate and prove value before committing to a choice. Since it
will be a separate artifact from the start we can always move the artifact
to a separate repo later (but moving the other way is harder).
* Users will get the sidecar "for free" when they install the daemon, they
don't need to take affirmative action to e.g. be able to restart their
cluster, run repair, or back their data up; it just comes out of the box
for free.

Unique pros of a separate repository sidecar:
* We can use a more modern build system like gradle instead of ant
* Merging changes is less "scary" I guess (I feel like if you're not
touching the daemon this is already true but I could see this being less
worrisome for some).
* Releasing a separate artifact is somewhat easier from a separate repo
(especially if we have gradle which makes e.g. building debs and rpms
trivial).
* We could backport to previous versions without getting into arguments
about bug fixes vs features.
* Committers could be different from the main repo, which ... may be a
useful thing

Non unique pros of a sidecar (could be achieved in the main repo or in a
separate repo):
* A separate build artifact .jar/.deb/.rpm that can be installed
separately. It's slightly easier with a separate repo but certainly not out
of reach within a single repo (indeed the current patch already creates a
separate jar, and we could create a separate .deb reasonably easily).
Personally I think having a separate .deb/.rpm is premature at this point
(for companies that really want it they can build their own packages using
the .jars), but I think it really is a distracting issue from where the
patch should go as we can always choose to remove experimental .jar files
that the main daemon doesn't touch.
* A separate process lifecycle. No matter where the sidecar goes, we get
the benefit of restarting it being less dangerous for availability than
restarting the main daemon.

That all being said, these are strong opinions weakly held and I would
rather get something actually committed so that we can prove value one way
or the other and am therefore, of course, happy to put sidecar patches
wherever someone can review and commit it.

-Joey

On Mon, Aug 20, 2018 at 1:52 PM sankalp kohli <ko...@gmail.com>
wrote:

> Hi,
>     I am starting a new thread to get consensus on where the side car
> should be contributed.
>
> Please send your responses with pro/cons of each approach or any other
> approach. Please be clear which approach you will pick while still giving
> pros/cons of both approaches.
>
> Thanks.
> Sankalp
>