You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Nate McCall <zz...@gmail.com> on 2020/02/18 21:21:46 UTC
[DISCUSS] Client protocol changes (Was: 20200217 4.0 Status Update)
[Moving to new message thread]
Thanks for bringing this up, Jordan.
IIRC, this was more a convention than a technical reason. Though I could be
completely misremembering this.
---------- Forwarded message ---------
From: Jordan West <jo...@gmail.com>
Date: Wed, Feb 19, 2020 at 10:13 AM
Subject: Re: 20200217 4.0 Status Update
To: <de...@cassandra.apache.org>
On Mon, Feb 17, 2020 at 12:52 PM Jeff Jirsa <jj...@gmail.com> wrote:
>
> beyond the client proto change being painful for anything other than major
> releases
>
>
This came up during the community meeting today and I wanted to bring a
question about it to the list: could someone who is *very* familiar with
the client proto share w/ the list why changing the proto in anything other
than a major release is so difficult? I hear this a lot and it seems to be
fact. So that all of us don't have to go read the code, a brief summary
would be super helpful. Or if there is a ticket that already covers this
even better! I'd also be curious if there have ever been any thoughts to
address it as it seems to be a consistent hurdle during the release cycle
and one that tends to further increase scope.
Thanks,
Jordan
>
>
> > On Feb 17, 2020, at 12:43 PM, Jon Meredith <jm...@gmail.com>
> wrote:
> >
> > My turn to give an update on 4.0 status. The 4.0 board created by Josh
> can
> > be found at
> >
> >
> > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355.
> >
> >
> > We have 94 unresolved tickets marked against the 4.0 release. [1]
> >
> >
> > Things seem to have settled into a phase of working to resolve issues,
> with
> > few new issues added.
> >
> >
> > 2 new tickets opened (that are marked against 4.0)
> >
> > 11 tickets closed (including one of the newly opened ones)
> >
> > 39 tickets received updates to JIRA of some kind in the last week
> >
> >
> > Cumulative flow over the last couple of weeks shows todo reducing and
> done
> > increasing as it should as we continue to close out work for the
release.
> >
> >
> >
>
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=14
> >
> >
> > Notables
> >
> > - Python 3 support for cqlsh has been committed (thank you all who
> > persevered on this)
> >
> > - Some activity on Windows support - perhaps not dead yet.
> >
> > - Lots of movement on documentation
> >
> > - Lots of activity on flaky tests.
> >
> > - Oldest ticket with a patch award goes to CASSANDRA-2848
> >
> >
> > There are 18 tickets marked as patch available (easy access from the
> > Dashboard [2], apologies if they're already picked up for review)
> >
> >
> > CASSANDRA-15567 Allow EXTRA_CLASSPATH to work in tarball/source
> > installations
> >
> > CASSANDRA-15553 Preview repair should include sstables from finalized
> > incremental repair sessions
> >
> > CASSANDRA-15550 Fix flaky test
> > org.apache.cassandra.streaming.StreamTransferTaskTest
> > testFailSessionDuringTransferShouldNotReleaseReferences
> >
> > CASSANDRA-15488/CASSANDRA-15353 Configuration file
> >
> > CASSANDRA-15484/CASSANDRA-15353 Read Repair
> >
> > CASSANDRA-15482/CASSANDRA-15353 Guarantees
> >
> > CASSANDRA-15481/CASSANDRA-15353 Data Modeling
> >
> > CASSANDRA-15393/CASSANDRA-15387 Add byte array backed cells
> >
> > CASSANDRA-15391/CASSANDRA-15387 Reduce heap footprint of commonly
> allocated
> > objects
> >
> > CASSANDRA-15367 Memtable memory allocations may deadlock
> >
> > CASSANDRA-15308 Fix flakey testAcquireReleaseOutbound -
> > org.apache.cassandra.net.ConnectionTest
> >
> > CASSANDRA-1530 5Fix multi DC nodetool status output
> >
> > CASSANDRA-14973 Bring v5 driver out of beta, introduce v6 before 4.0
> > release is cut
> >
> > CASSANDRA-14939 fix some operational holes in incremental repair
> >
> > CASSANDRA-14904 SSTableloader doesn't understand listening for CQL
> > connections on multiple ports
> >
> > CASSANDRA-14842 SSL connection problems when upgrading to 4.0 when
> > upgrading from 3.0.x
> >
> > CASSANDRA-14761 Rename speculative_retry to match
additional_write_policy
> >
> > CASSANDRA-2848 Make the Client API support passing down timeouts
> >
> >
> > *LHF / Failing Tests*: We have 7 unassigned test failures that are all
> >
> > great candidates to pick up and get involved in:
> >
> >
>
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&quickFilter=1660&quickFilter=1661&quickFilter=1658
> >
> >
> > Thanks again to everybody for all the contributions. It's really good to
> > see the open issue count start dropping.
> >
> >
> > Feedback on whether this information is useful and how it can be
improved
> > is both welcome and appreciated.
> >
> >
> > Cheers, Jon
> >
> >
> > [1] Unresolved 4.0 tickets
> >
>
https://issues.apache.org/jira/browse/CASSANDRA-15567?filter=12347782&jql=project%20%3D%20cassandra%20AND%20fixversion%20in%20(4.0%2C%204.0.0%2C%204.0-alpha%2C%204.0-beta)%20AND%20status%20!%3D%20Resolved
> >
> > [2] Patch Available
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12334910
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] Client protocol changes (Was: 20200217 4.0 Status Update)
Posted by Yifan Cai <yc...@gmail.com>.
CustomPayload should be used to provide customization via a custom query
handler (that is outside of Cassandra source).
Supporting custom timeout per query is a new feature. It is more clear to
assign a dedicated query flag. In V5, the available number of query flags
expanded from 8 (in V4 and prior) to 32. We do not need to piggyback the
custom payload. Using a dedicated flag gives the same forward and backward
compatibility as custom payload.
On Tue, Feb 18, 2020 at 2:28 PM David Capwell <dc...@gmail.com> wrote:
> Given the JIRA in question, if you want to override the timeout to lower
> it, then the worst case if not supported yet is that you get the default
> timeout. So this then makes me wonder "is there a way to add metadata to a
> message which is ignored if unknown" (aka forward compatibility). Skimming
> the frame code i see we have
>
> boolean isCustomPayload =
> frame.header.flags.contains(Frame.Header.Flag.CUSTOM_PAYLOAD);
> boolean hasWarning =
> frame.header.flags.contains(Frame.Header.Flag.WARNING);
>
> UUID tracingId = isRequest || !isTracing ? null :
> CBUtil.readUUID(frame.body);
> List<String> warnings = isRequest || !hasWarning ? null :
> CBUtil.readStringList(frame.body);
> Map<String, ByteBuffer> customPayload = !isCustomPayload ? null :
> CBUtil.readBytesMap(frame.body);
>
> This makes me wonder if we could picky back off that for new features, that
> way older servers just ignore them. I have no idea of the negatives of
> customPayload (other than strings are more bytes for messages, evolution
> may be based off key names so annoying, etc.), but tags which are ignored
> sounds promising
>
>
> On Tue, Feb 18, 2020 at 1:53 PM Jeff Jirsa <jj...@gmail.com> wrote:
>
> > A few notes:
> >
> > - Protocol changes add work to the rest of the ecosystem. Drivers have to
> > update, etc.
> > - Nobody expects protocol changes in minors, though it's less of a
> concern
> > if we don't deprecate out the older version. E.g. if 4.0 launches with
> > protocol v4 and protocol v5, and then 4.0.2 adds protocol v6, do we
> > deprecate out v4? If yes, you potentially break clients that only
> supported
> > v3 and v4 in a minor version upgrade, which is unexpected. If not, how
> many
> > protocol versions are you willing to support at any given time?
> > - Having protocol changes introduces risk. Paging behavior across
> protocol
> > versions is the site of a number of different bugs recently.
> >
> >
> > On Tue, Feb 18, 2020 at 1:46 PM Tolbert, Andrew <x...@andrewtolbert.com>
> > wrote:
> >
> > > I don't know the technical answer, but I suspect two motivations for
> > > doing new protocol versions in major releases could include:
> > >
> > > * protocol changes can be tied to feature changes that typically come
> > > in a major release.
> > > * protocol changes should be as infrequent as major releases. Each
> > > new protocol version is another thing in the test matrix that needs to
> > > be tested.
> > >
> > > That last point can make it hard to get new changes in. If something
> > > doesn't make the upcoming protocol version, it might be years before
> > > another one, but I also think it's worth it to do this infrequently as
> > > it makes maintaining client and server code easier if there are less
> > > protocol versions to worry about.
> > >
> > > On the client-side, libraries themselves should be avoiding making
> > > Cassandra version checks when detecting capabilities. There are a few
> > > exceptions, such as system table parsing for schema & peers,
> > > but those aren't related to the protocol.
> > >
> > > Thanks,
> > > Andy
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Feb 18, 2020 at 1:22 PM Nate McCall <zz...@gmail.com>
> wrote:
> > > >
> > > > [Moving to new message thread]
> > > >
> > > > Thanks for bringing this up, Jordan.
> > > >
> > > > IIRC, this was more a convention than a technical reason. Though I
> > could
> > > be
> > > > completely misremembering this.
> > > >
> > > > ---------- Forwarded message ---------
> > > > From: Jordan West <jo...@gmail.com>
> > > > Date: Wed, Feb 19, 2020 at 10:13 AM
> > > > Subject: Re: 20200217 4.0 Status Update
> > > > To: <de...@cassandra.apache.org>
> > > >
> > > >
> > > > On Mon, Feb 17, 2020 at 12:52 PM Jeff Jirsa <jj...@gmail.com>
> wrote:
> > > >
> > > > >
> > > > > beyond the client proto change being painful for anything other
> than
> > > major
> > > > > releases
> > > > >
> > > > >
> > > > This came up during the community meeting today and I wanted to
> bring a
> > > > question about it to the list: could someone who is *very* familiar
> > with
> > > > the client proto share w/ the list why changing the proto in anything
> > > other
> > > > than a major release is so difficult? I hear this a lot and it seems
> to
> > > be
> > > > fact. So that all of us don't have to go read the code, a brief
> summary
> > > > would be super helpful. Or if there is a ticket that already covers
> > this
> > > > even better! I'd also be curious if there have ever been any thoughts
> > to
> > > > address it as it seems to be a consistent hurdle during the release
> > cycle
> > > > and one that tends to further increase scope.
> > > >
> > > > Thanks,
> > > > Jordan
> > > >
> > > > >
> > > > >
> > > > > > On Feb 17, 2020, at 12:43 PM, Jon Meredith <
> jmeredithco@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > My turn to give an update on 4.0 status. The 4.0 board created
> by
> > > Josh
> > > > > can
> > > > > > be found at
> > > > > >
> > > > > >
> > > > > >
> > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355.
> > > > > >
> > > > > >
> > > > > > We have 94 unresolved tickets marked against the 4.0 release. [1]
> > > > > >
> > > > > >
> > > > > > Things seem to have settled into a phase of working to resolve
> > > issues,
> > > > > with
> > > > > > few new issues added.
> > > > > >
> > > > > >
> > > > > > 2 new tickets opened (that are marked against 4.0)
> > > > > >
> > > > > > 11 tickets closed (including one of the newly opened ones)
> > > > > >
> > > > > > 39 tickets received updates to JIRA of some kind in the last week
> > > > > >
> > > > > >
> > > > > > Cumulative flow over the last couple of weeks shows todo reducing
> > and
> > > > > done
> > > > > > increasing as it should as we continue to close out work for the
> > > > release.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=14
> > > > > >
> > > > > >
> > > > > > Notables
> > > > > >
> > > > > > - Python 3 support for cqlsh has been committed (thank you all
> who
> > > > > > persevered on this)
> > > > > >
> > > > > > - Some activity on Windows support - perhaps not dead yet.
> > > > > >
> > > > > > - Lots of movement on documentation
> > > > > >
> > > > > > - Lots of activity on flaky tests.
> > > > > >
> > > > > > - Oldest ticket with a patch award goes to CASSANDRA-2848
> > > > > >
> > > > > >
> > > > > > There are 18 tickets marked as patch available (easy access from
> > the
> > > > > > Dashboard [2], apologies if they're already picked up for review)
> > > > > >
> > > > > >
> > > > > > CASSANDRA-15567 Allow EXTRA_CLASSPATH to work in tarball/source
> > > > > > installations
> > > > > >
> > > > > > CASSANDRA-15553 Preview repair should include sstables from
> > finalized
> > > > > > incremental repair sessions
> > > > > >
> > > > > > CASSANDRA-15550 Fix flaky test
> > > > > > org.apache.cassandra.streaming.StreamTransferTaskTest
> > > > > > testFailSessionDuringTransferShouldNotReleaseReferences
> > > > > >
> > > > > > CASSANDRA-15488/CASSANDRA-15353 Configuration file
> > > > > >
> > > > > > CASSANDRA-15484/CASSANDRA-15353 Read Repair
> > > > > >
> > > > > > CASSANDRA-15482/CASSANDRA-15353 Guarantees
> > > > > >
> > > > > > CASSANDRA-15481/CASSANDRA-15353 Data Modeling
> > > > > >
> > > > > > CASSANDRA-15393/CASSANDRA-15387 Add byte array backed cells
> > > > > >
> > > > > > CASSANDRA-15391/CASSANDRA-15387 Reduce heap footprint of commonly
> > > > > allocated
> > > > > > objects
> > > > > >
> > > > > > CASSANDRA-15367 Memtable memory allocations may deadlock
> > > > > >
> > > > > > CASSANDRA-15308 Fix flakey testAcquireReleaseOutbound -
> > > > > > org.apache.cassandra.net.ConnectionTest
> > > > > >
> > > > > > CASSANDRA-1530 5Fix multi DC nodetool status output
> > > > > >
> > > > > > CASSANDRA-14973 Bring v5 driver out of beta, introduce v6 before
> > 4.0
> > > > > > release is cut
> > > > > >
> > > > > > CASSANDRA-14939 fix some operational holes in incremental repair
> > > > > >
> > > > > > CASSANDRA-14904 SSTableloader doesn't understand listening for
> CQL
> > > > > > connections on multiple ports
> > > > > >
> > > > > > CASSANDRA-14842 SSL connection problems when upgrading to 4.0
> when
> > > > > > upgrading from 3.0.x
> > > > > >
> > > > > > CASSANDRA-14761 Rename speculative_retry to match
> > > > additional_write_policy
> > > > > >
> > > > > > CASSANDRA-2848 Make the Client API support passing down timeouts
> > > > > >
> > > > > >
> > > > > > *LHF / Failing Tests*: We have 7 unassigned test failures that
> are
> > > all
> > > > > >
> > > > > > great candidates to pick up and get involved in:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&quickFilter=1660&quickFilter=1661&quickFilter=1658
> > > > > >
> > > > > >
> > > > > > Thanks again to everybody for all the contributions. It's really
> > > good to
> > > > > > see the open issue count start dropping.
> > > > > >
> > > > > >
> > > > > > Feedback on whether this information is useful and how it can be
> > > > improved
> > > > > > is both welcome and appreciated.
> > > > > >
> > > > > >
> > > > > > Cheers, Jon
> > > > > >
> > > > > >
> > > > > > [1] Unresolved 4.0 tickets
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/CASSANDRA-15567?filter=12347782&jql=project%20%3D%20cassandra%20AND%20fixversion%20in%20(4.0%2C%204.0.0%2C%204.0-alpha%2C%204.0-beta)%20AND%20status%20!%3D%20Resolved
> > > > > >
> > > > > > [2] Patch Available
> > > > > >
> > > > >
> > >
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12334910
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] Client protocol changes (Was: 20200217 4.0 Status Update)
Posted by David Capwell <dc...@gmail.com>.
Given the JIRA in question, if you want to override the timeout to lower
it, then the worst case if not supported yet is that you get the default
timeout. So this then makes me wonder "is there a way to add metadata to a
message which is ignored if unknown" (aka forward compatibility). Skimming
the frame code i see we have
boolean isCustomPayload =
frame.header.flags.contains(Frame.Header.Flag.CUSTOM_PAYLOAD);
boolean hasWarning = frame.header.flags.contains(Frame.Header.Flag.WARNING);
UUID tracingId = isRequest || !isTracing ? null : CBUtil.readUUID(frame.body);
List<String> warnings = isRequest || !hasWarning ? null :
CBUtil.readStringList(frame.body);
Map<String, ByteBuffer> customPayload = !isCustomPayload ? null :
CBUtil.readBytesMap(frame.body);
This makes me wonder if we could picky back off that for new features, that
way older servers just ignore them. I have no idea of the negatives of
customPayload (other than strings are more bytes for messages, evolution
may be based off key names so annoying, etc.), but tags which are ignored
sounds promising
On Tue, Feb 18, 2020 at 1:53 PM Jeff Jirsa <jj...@gmail.com> wrote:
> A few notes:
>
> - Protocol changes add work to the rest of the ecosystem. Drivers have to
> update, etc.
> - Nobody expects protocol changes in minors, though it's less of a concern
> if we don't deprecate out the older version. E.g. if 4.0 launches with
> protocol v4 and protocol v5, and then 4.0.2 adds protocol v6, do we
> deprecate out v4? If yes, you potentially break clients that only supported
> v3 and v4 in a minor version upgrade, which is unexpected. If not, how many
> protocol versions are you willing to support at any given time?
> - Having protocol changes introduces risk. Paging behavior across protocol
> versions is the site of a number of different bugs recently.
>
>
> On Tue, Feb 18, 2020 at 1:46 PM Tolbert, Andrew <x...@andrewtolbert.com>
> wrote:
>
> > I don't know the technical answer, but I suspect two motivations for
> > doing new protocol versions in major releases could include:
> >
> > * protocol changes can be tied to feature changes that typically come
> > in a major release.
> > * protocol changes should be as infrequent as major releases. Each
> > new protocol version is another thing in the test matrix that needs to
> > be tested.
> >
> > That last point can make it hard to get new changes in. If something
> > doesn't make the upcoming protocol version, it might be years before
> > another one, but I also think it's worth it to do this infrequently as
> > it makes maintaining client and server code easier if there are less
> > protocol versions to worry about.
> >
> > On the client-side, libraries themselves should be avoiding making
> > Cassandra version checks when detecting capabilities. There are a few
> > exceptions, such as system table parsing for schema & peers,
> > but those aren't related to the protocol.
> >
> > Thanks,
> > Andy
> >
> >
> >
> >
> >
> > On Tue, Feb 18, 2020 at 1:22 PM Nate McCall <zz...@gmail.com> wrote:
> > >
> > > [Moving to new message thread]
> > >
> > > Thanks for bringing this up, Jordan.
> > >
> > > IIRC, this was more a convention than a technical reason. Though I
> could
> > be
> > > completely misremembering this.
> > >
> > > ---------- Forwarded message ---------
> > > From: Jordan West <jo...@gmail.com>
> > > Date: Wed, Feb 19, 2020 at 10:13 AM
> > > Subject: Re: 20200217 4.0 Status Update
> > > To: <de...@cassandra.apache.org>
> > >
> > >
> > > On Mon, Feb 17, 2020 at 12:52 PM Jeff Jirsa <jj...@gmail.com> wrote:
> > >
> > > >
> > > > beyond the client proto change being painful for anything other than
> > major
> > > > releases
> > > >
> > > >
> > > This came up during the community meeting today and I wanted to bring a
> > > question about it to the list: could someone who is *very* familiar
> with
> > > the client proto share w/ the list why changing the proto in anything
> > other
> > > than a major release is so difficult? I hear this a lot and it seems to
> > be
> > > fact. So that all of us don't have to go read the code, a brief summary
> > > would be super helpful. Or if there is a ticket that already covers
> this
> > > even better! I'd also be curious if there have ever been any thoughts
> to
> > > address it as it seems to be a consistent hurdle during the release
> cycle
> > > and one that tends to further increase scope.
> > >
> > > Thanks,
> > > Jordan
> > >
> > > >
> > > >
> > > > > On Feb 17, 2020, at 12:43 PM, Jon Meredith <jm...@gmail.com>
> > > > wrote:
> > > > >
> > > > > My turn to give an update on 4.0 status. The 4.0 board created by
> > Josh
> > > > can
> > > > > be found at
> > > > >
> > > > >
> > > > >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355.
> > > > >
> > > > >
> > > > > We have 94 unresolved tickets marked against the 4.0 release. [1]
> > > > >
> > > > >
> > > > > Things seem to have settled into a phase of working to resolve
> > issues,
> > > > with
> > > > > few new issues added.
> > > > >
> > > > >
> > > > > 2 new tickets opened (that are marked against 4.0)
> > > > >
> > > > > 11 tickets closed (including one of the newly opened ones)
> > > > >
> > > > > 39 tickets received updates to JIRA of some kind in the last week
> > > > >
> > > > >
> > > > > Cumulative flow over the last couple of weeks shows todo reducing
> and
> > > > done
> > > > > increasing as it should as we continue to close out work for the
> > > release.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=14
> > > > >
> > > > >
> > > > > Notables
> > > > >
> > > > > - Python 3 support for cqlsh has been committed (thank you all who
> > > > > persevered on this)
> > > > >
> > > > > - Some activity on Windows support - perhaps not dead yet.
> > > > >
> > > > > - Lots of movement on documentation
> > > > >
> > > > > - Lots of activity on flaky tests.
> > > > >
> > > > > - Oldest ticket with a patch award goes to CASSANDRA-2848
> > > > >
> > > > >
> > > > > There are 18 tickets marked as patch available (easy access from
> the
> > > > > Dashboard [2], apologies if they're already picked up for review)
> > > > >
> > > > >
> > > > > CASSANDRA-15567 Allow EXTRA_CLASSPATH to work in tarball/source
> > > > > installations
> > > > >
> > > > > CASSANDRA-15553 Preview repair should include sstables from
> finalized
> > > > > incremental repair sessions
> > > > >
> > > > > CASSANDRA-15550 Fix flaky test
> > > > > org.apache.cassandra.streaming.StreamTransferTaskTest
> > > > > testFailSessionDuringTransferShouldNotReleaseReferences
> > > > >
> > > > > CASSANDRA-15488/CASSANDRA-15353 Configuration file
> > > > >
> > > > > CASSANDRA-15484/CASSANDRA-15353 Read Repair
> > > > >
> > > > > CASSANDRA-15482/CASSANDRA-15353 Guarantees
> > > > >
> > > > > CASSANDRA-15481/CASSANDRA-15353 Data Modeling
> > > > >
> > > > > CASSANDRA-15393/CASSANDRA-15387 Add byte array backed cells
> > > > >
> > > > > CASSANDRA-15391/CASSANDRA-15387 Reduce heap footprint of commonly
> > > > allocated
> > > > > objects
> > > > >
> > > > > CASSANDRA-15367 Memtable memory allocations may deadlock
> > > > >
> > > > > CASSANDRA-15308 Fix flakey testAcquireReleaseOutbound -
> > > > > org.apache.cassandra.net.ConnectionTest
> > > > >
> > > > > CASSANDRA-1530 5Fix multi DC nodetool status output
> > > > >
> > > > > CASSANDRA-14973 Bring v5 driver out of beta, introduce v6 before
> 4.0
> > > > > release is cut
> > > > >
> > > > > CASSANDRA-14939 fix some operational holes in incremental repair
> > > > >
> > > > > CASSANDRA-14904 SSTableloader doesn't understand listening for CQL
> > > > > connections on multiple ports
> > > > >
> > > > > CASSANDRA-14842 SSL connection problems when upgrading to 4.0 when
> > > > > upgrading from 3.0.x
> > > > >
> > > > > CASSANDRA-14761 Rename speculative_retry to match
> > > additional_write_policy
> > > > >
> > > > > CASSANDRA-2848 Make the Client API support passing down timeouts
> > > > >
> > > > >
> > > > > *LHF / Failing Tests*: We have 7 unassigned test failures that are
> > all
> > > > >
> > > > > great candidates to pick up and get involved in:
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&quickFilter=1660&quickFilter=1661&quickFilter=1658
> > > > >
> > > > >
> > > > > Thanks again to everybody for all the contributions. It's really
> > good to
> > > > > see the open issue count start dropping.
> > > > >
> > > > >
> > > > > Feedback on whether this information is useful and how it can be
> > > improved
> > > > > is both welcome and appreciated.
> > > > >
> > > > >
> > > > > Cheers, Jon
> > > > >
> > > > >
> > > > > [1] Unresolved 4.0 tickets
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/CASSANDRA-15567?filter=12347782&jql=project%20%3D%20cassandra%20AND%20fixversion%20in%20(4.0%2C%204.0.0%2C%204.0-alpha%2C%204.0-beta)%20AND%20status%20!%3D%20Resolved
> > > > >
> > > > > [2] Patch Available
> > > > >
> > > >
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12334910
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
>
Re: [DISCUSS] Client protocol changes (Was: 20200217 4.0 Status Update)
Posted by Jorge Bay Gondra <jo...@gmail.com>.
There were some discussions on Slack whether CASSANDRA-2848 should be in
scope for 4.0 or not, given that its an enhancement.
I'm +1 on descoping it and leave it for a later 4.x
As it involves a change in the protocol, we could use non-breaking changes
to add these types of features to protocol_v5 in 4.x without a protocol
version bump (new flag + new SUPPORTED option), like I've proposed above.
On Thu, Feb 20, 2020 at 2:52 PM Aleksey Yeshchenko
<al...@apple.com.invalid> wrote:
> For what it’s worth, we could trivially implement support for passing down
> the timeout in 4.0.0, so that both the server and the client are able to
> parse the frames with and without them, but delay implementation of the
> server side logic necessary for terminating requests early until a further
> minor (4.1/4.0.1).
>
> > On 19 Feb 2020, at 15:39, Jorge Bay Gondra <jo...@gmail.com>
> wrote:
> >
> > Also worth mentioning that, from the driver's perspective, it has to
> > support a protocol version during the lifetime of the C* version line.
> For
> > example, the drivers should drop support for protocol v3 after C* 2.1
> goes
> > EOL, somewhere this year, a protocol that was released back in 2014.
> >
> > We _could_ establish looser restrictions on whats a breaking change in a
> > protocol version (needing a version bump), that way the driver can
> support
> > a protocol version partially and a protocol version could evolve within
> > those limits.
> >
> > Back to the query timeout, a new query flag that can only be set by the
> > client is not a breaking change for the driver. The driver could ask
> > whether that feature of the protocol v5 is supported (OPTIONS/SUPPORTED
> > messages), without having to identify the server version.
> >
> > On Wed, Feb 19, 2020 at 12:24 AM Benedict Elliott Smith <
> benedict@apache.org>
> > wrote:
> >
> >> Behaviours don't have to be switched only with a new protocol version;
> >> it's possible to support optional feature/modifier flags, the support
> for
> >> which is negotiated with a client on connection.
> >>
> >> A protocol version change seems reasonable to limit to major releases,
> but
> >> a protocol feature seems perfectly reasonable to introduce in a minor, I
> >> think? Ideally a version change would only be necessary for forced
> >> deprecation/standardisation of features, behaviour and stream encodings.
> >>
> >>
> >> On 18/02/2020, 21:53, "Jeff Jirsa" <jj...@gmail.com> wrote:
> >>
> >> A few notes:
> >>
> >> - Protocol changes add work to the rest of the ecosystem. Drivers
> have
> >> to
> >> update, etc.
> >> - Nobody expects protocol changes in minors, though it's less of a
> >> concern
> >> if we don't deprecate out the older version. E.g. if 4.0 launches
> with
> >> protocol v4 and protocol v5, and then 4.0.2 adds protocol v6, do we
> >> deprecate out v4? If yes, you potentially break clients that only
> >> supported
> >> v3 and v4 in a minor version upgrade, which is unexpected. If not,
> how
> >> many
> >> protocol versions are you willing to support at any given time?
> >> - Having protocol changes introduces risk. Paging behavior across
> >> protocol
> >> versions is the site of a number of different bugs recently.
> >>
> >>
> >> On Tue, Feb 18, 2020 at 1:46 PM Tolbert, Andrew <x@andrewtolbert.com
> >
> >> wrote:
> >>
> >>> I don't know the technical answer, but I suspect two motivations for
> >>> doing new protocol versions in major releases could include:
> >>>
> >>> * protocol changes can be tied to feature changes that typically come
> >>> in a major release.
> >>> * protocol changes should be as infrequent as major releases. Each
> >>> new protocol version is another thing in the test matrix that needs
> >> to
> >>> be tested.
> >>>
> >>> That last point can make it hard to get new changes in. If something
> >>> doesn't make the upcoming protocol version, it might be years before
> >>> another one, but I also think it's worth it to do this infrequently
> >> as
> >>> it makes maintaining client and server code easier if there are less
> >>> protocol versions to worry about.
> >>>
> >>> On the client-side, libraries themselves should be avoiding making
> >>> Cassandra version checks when detecting capabilities. There are a
> >> few
> >>> exceptions, such as system table parsing for schema & peers,
> >>> but those aren't related to the protocol.
> >>>
> >>> Thanks,
> >>> Andy
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Tue, Feb 18, 2020 at 1:22 PM Nate McCall <zz...@gmail.com>
> >> wrote:
> >>>>
> >>>> [Moving to new message thread]
> >>>>
> >>>> Thanks for bringing this up, Jordan.
> >>>>
> >>>> IIRC, this was more a convention than a technical reason. Though I
> >> could
> >>> be
> >>>> completely misremembering this.
> >>>>
> >>>> ---------- Forwarded message ---------
> >>>> From: Jordan West <jo...@gmail.com>
> >>>> Date: Wed, Feb 19, 2020 at 10:13 AM
> >>>> Subject: Re: 20200217 4.0 Status Update
> >>>> To: <de...@cassandra.apache.org>
> >>>>
> >>>>
> >>>> On Mon, Feb 17, 2020 at 12:52 PM Jeff Jirsa <jj...@gmail.com>
> >> wrote:
> >>>>
> >>>>>
> >>>>> beyond the client proto change being painful for anything other
> >> than
> >>> major
> >>>>> releases
> >>>>>
> >>>>>
> >>>> This came up during the community meeting today and I wanted to
> >> bring a
> >>>> question about it to the list: could someone who is *very*
> >> familiar with
> >>>> the client proto share w/ the list why changing the proto in
> >> anything
> >>> other
> >>>> than a major release is so difficult? I hear this a lot and it
> >> seems to
> >>> be
> >>>> fact. So that all of us don't have to go read the code, a brief
> >> summary
> >>>> would be super helpful. Or if there is a ticket that already
> >> covers this
> >>>> even better! I'd also be curious if there have ever been any
> >> thoughts to
> >>>> address it as it seems to be a consistent hurdle during the
> >> release cycle
> >>>> and one that tends to further increase scope.
> >>>>
> >>>> Thanks,
> >>>> Jordan
> >>>>
> >>>>>
> >>>>>
> >>>>>> On Feb 17, 2020, at 12:43 PM, Jon Meredith <
> >> jmeredithco@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> My turn to give an update on 4.0 status. The 4.0 board
> >> created by
> >>> Josh
> >>>>> can
> >>>>>> be found at
> >>>>>>
> >>>>>>
> >>>>>>
> >> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355.
> >>>>>>
> >>>>>>
> >>>>>> We have 94 unresolved tickets marked against the 4.0 release.
> >> [1]
> >>>>>>
> >>>>>>
> >>>>>> Things seem to have settled into a phase of working to resolve
> >>> issues,
> >>>>> with
> >>>>>> few new issues added.
> >>>>>>
> >>>>>>
> >>>>>> 2 new tickets opened (that are marked against 4.0)
> >>>>>>
> >>>>>> 11 tickets closed (including one of the newly opened ones)
> >>>>>>
> >>>>>> 39 tickets received updates to JIRA of some kind in the last
> >> week
> >>>>>>
> >>>>>>
> >>>>>> Cumulative flow over the last couple of weeks shows todo
> >> reducing and
> >>>>> done
> >>>>>> increasing as it should as we continue to close out work for
> >> the
> >>>> release.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=14
> >>>>>>
> >>>>>>
> >>>>>> Notables
> >>>>>>
> >>>>>> - Python 3 support for cqlsh has been committed (thank you all
> >> who
> >>>>>> persevered on this)
> >>>>>>
> >>>>>> - Some activity on Windows support - perhaps not dead yet.
> >>>>>>
> >>>>>> - Lots of movement on documentation
> >>>>>>
> >>>>>> - Lots of activity on flaky tests.
> >>>>>>
> >>>>>> - Oldest ticket with a patch award goes to CASSANDRA-2848
> >>>>>>
> >>>>>>
> >>>>>> There are 18 tickets marked as patch available (easy access
> >> from the
> >>>>>> Dashboard [2], apologies if they're already picked up for
> >> review)
> >>>>>>
> >>>>>>
> >>>>>> CASSANDRA-15567 Allow EXTRA_CLASSPATH to work in tarball/source
> >>>>>> installations
> >>>>>>
> >>>>>> CASSANDRA-15553 Preview repair should include sstables from
> >> finalized
> >>>>>> incremental repair sessions
> >>>>>>
> >>>>>> CASSANDRA-15550 Fix flaky test
> >>>>>> org.apache.cassandra.streaming.StreamTransferTaskTest
> >>>>>> testFailSessionDuringTransferShouldNotReleaseReferences
> >>>>>>
> >>>>>> CASSANDRA-15488/CASSANDRA-15353 Configuration file
> >>>>>>
> >>>>>> CASSANDRA-15484/CASSANDRA-15353 Read Repair
> >>>>>>
> >>>>>> CASSANDRA-15482/CASSANDRA-15353 Guarantees
> >>>>>>
> >>>>>> CASSANDRA-15481/CASSANDRA-15353 Data Modeling
> >>>>>>
> >>>>>> CASSANDRA-15393/CASSANDRA-15387 Add byte array backed cells
> >>>>>>
> >>>>>> CASSANDRA-15391/CASSANDRA-15387 Reduce heap footprint of
> >> commonly
> >>>>> allocated
> >>>>>> objects
> >>>>>>
> >>>>>> CASSANDRA-15367 Memtable memory allocations may deadlock
> >>>>>>
> >>>>>> CASSANDRA-15308 Fix flakey testAcquireReleaseOutbound -
> >>>>>> org.apache.cassandra.net.ConnectionTest
> >>>>>>
> >>>>>> CASSANDRA-1530 5Fix multi DC nodetool status output
> >>>>>>
> >>>>>> CASSANDRA-14973 Bring v5 driver out of beta, introduce v6
> >> before 4.0
> >>>>>> release is cut
> >>>>>>
> >>>>>> CASSANDRA-14939 fix some operational holes in incremental
> >> repair
> >>>>>>
> >>>>>> CASSANDRA-14904 SSTableloader doesn't understand listening for
> >> CQL
> >>>>>> connections on multiple ports
> >>>>>>
> >>>>>> CASSANDRA-14842 SSL connection problems when upgrading to 4.0
> >> when
> >>>>>> upgrading from 3.0.x
> >>>>>>
> >>>>>> CASSANDRA-14761 Rename speculative_retry to match
> >>>> additional_write_policy
> >>>>>>
> >>>>>> CASSANDRA-2848 Make the Client API support passing down
> >> timeouts
> >>>>>>
> >>>>>>
> >>>>>> *LHF / Failing Tests*: We have 7 unassigned test failures that
> >> are
> >>> all
> >>>>>>
> >>>>>> great candidates to pick up and get involved in:
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&quickFilter=1660&quickFilter=1661&quickFilter=1658
> >>>>>>
> >>>>>>
> >>>>>> Thanks again to everybody for all the contributions. It's
> >> really
> >>> good to
> >>>>>> see the open issue count start dropping.
> >>>>>>
> >>>>>>
> >>>>>> Feedback on whether this information is useful and how it can
> >> be
> >>>> improved
> >>>>>> is both welcome and appreciated.
> >>>>>>
> >>>>>>
> >>>>>> Cheers, Jon
> >>>>>>
> >>>>>>
> >>>>>> [1] Unresolved 4.0 tickets
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://issues.apache.org/jira/browse/CASSANDRA-15567?filter=12347782&jql=project%20%3D%20cassandra%20AND%20fixversion%20in%20(4.0%2C%204.0.0%2C%204.0-alpha%2C%204.0-beta)%20AND%20status%20!%3D%20Resolved
> >>>>>>
> >>>>>> [2] Patch Available
> >>>>>>
> >>>>>
> >>>
> >>
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12334910
> >>>>>
> >>>>>
> >> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>>>
> >>>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >>> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>>
> >>>
> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> >> For additional commands, e-mail: dev-help@cassandra.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] Client protocol changes (Was: 20200217 4.0 Status Update)
Posted by Aleksey Yeshchenko <al...@apple.com.INVALID>.
For what it’s worth, we could trivially implement support for passing down the timeout in 4.0.0, so that both the server and the client are able to parse the frames with and without them, but delay implementation of the server side logic necessary for terminating requests early until a further minor (4.1/4.0.1).
> On 19 Feb 2020, at 15:39, Jorge Bay Gondra <jo...@gmail.com> wrote:
>
> Also worth mentioning that, from the driver's perspective, it has to
> support a protocol version during the lifetime of the C* version line. For
> example, the drivers should drop support for protocol v3 after C* 2.1 goes
> EOL, somewhere this year, a protocol that was released back in 2014.
>
> We _could_ establish looser restrictions on whats a breaking change in a
> protocol version (needing a version bump), that way the driver can support
> a protocol version partially and a protocol version could evolve within
> those limits.
>
> Back to the query timeout, a new query flag that can only be set by the
> client is not a breaking change for the driver. The driver could ask
> whether that feature of the protocol v5 is supported (OPTIONS/SUPPORTED
> messages), without having to identify the server version.
>
> On Wed, Feb 19, 2020 at 12:24 AM Benedict Elliott Smith <be...@apache.org>
> wrote:
>
>> Behaviours don't have to be switched only with a new protocol version;
>> it's possible to support optional feature/modifier flags, the support for
>> which is negotiated with a client on connection.
>>
>> A protocol version change seems reasonable to limit to major releases, but
>> a protocol feature seems perfectly reasonable to introduce in a minor, I
>> think? Ideally a version change would only be necessary for forced
>> deprecation/standardisation of features, behaviour and stream encodings.
>>
>>
>> On 18/02/2020, 21:53, "Jeff Jirsa" <jj...@gmail.com> wrote:
>>
>> A few notes:
>>
>> - Protocol changes add work to the rest of the ecosystem. Drivers have
>> to
>> update, etc.
>> - Nobody expects protocol changes in minors, though it's less of a
>> concern
>> if we don't deprecate out the older version. E.g. if 4.0 launches with
>> protocol v4 and protocol v5, and then 4.0.2 adds protocol v6, do we
>> deprecate out v4? If yes, you potentially break clients that only
>> supported
>> v3 and v4 in a minor version upgrade, which is unexpected. If not, how
>> many
>> protocol versions are you willing to support at any given time?
>> - Having protocol changes introduces risk. Paging behavior across
>> protocol
>> versions is the site of a number of different bugs recently.
>>
>>
>> On Tue, Feb 18, 2020 at 1:46 PM Tolbert, Andrew <x...@andrewtolbert.com>
>> wrote:
>>
>>> I don't know the technical answer, but I suspect two motivations for
>>> doing new protocol versions in major releases could include:
>>>
>>> * protocol changes can be tied to feature changes that typically come
>>> in a major release.
>>> * protocol changes should be as infrequent as major releases. Each
>>> new protocol version is another thing in the test matrix that needs
>> to
>>> be tested.
>>>
>>> That last point can make it hard to get new changes in. If something
>>> doesn't make the upcoming protocol version, it might be years before
>>> another one, but I also think it's worth it to do this infrequently
>> as
>>> it makes maintaining client and server code easier if there are less
>>> protocol versions to worry about.
>>>
>>> On the client-side, libraries themselves should be avoiding making
>>> Cassandra version checks when detecting capabilities. There are a
>> few
>>> exceptions, such as system table parsing for schema & peers,
>>> but those aren't related to the protocol.
>>>
>>> Thanks,
>>> Andy
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Feb 18, 2020 at 1:22 PM Nate McCall <zz...@gmail.com>
>> wrote:
>>>>
>>>> [Moving to new message thread]
>>>>
>>>> Thanks for bringing this up, Jordan.
>>>>
>>>> IIRC, this was more a convention than a technical reason. Though I
>> could
>>> be
>>>> completely misremembering this.
>>>>
>>>> ---------- Forwarded message ---------
>>>> From: Jordan West <jo...@gmail.com>
>>>> Date: Wed, Feb 19, 2020 at 10:13 AM
>>>> Subject: Re: 20200217 4.0 Status Update
>>>> To: <de...@cassandra.apache.org>
>>>>
>>>>
>>>> On Mon, Feb 17, 2020 at 12:52 PM Jeff Jirsa <jj...@gmail.com>
>> wrote:
>>>>
>>>>>
>>>>> beyond the client proto change being painful for anything other
>> than
>>> major
>>>>> releases
>>>>>
>>>>>
>>>> This came up during the community meeting today and I wanted to
>> bring a
>>>> question about it to the list: could someone who is *very*
>> familiar with
>>>> the client proto share w/ the list why changing the proto in
>> anything
>>> other
>>>> than a major release is so difficult? I hear this a lot and it
>> seems to
>>> be
>>>> fact. So that all of us don't have to go read the code, a brief
>> summary
>>>> would be super helpful. Or if there is a ticket that already
>> covers this
>>>> even better! I'd also be curious if there have ever been any
>> thoughts to
>>>> address it as it seems to be a consistent hurdle during the
>> release cycle
>>>> and one that tends to further increase scope.
>>>>
>>>> Thanks,
>>>> Jordan
>>>>
>>>>>
>>>>>
>>>>>> On Feb 17, 2020, at 12:43 PM, Jon Meredith <
>> jmeredithco@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> My turn to give an update on 4.0 status. The 4.0 board
>> created by
>>> Josh
>>>>> can
>>>>>> be found at
>>>>>>
>>>>>>
>>>>>>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355.
>>>>>>
>>>>>>
>>>>>> We have 94 unresolved tickets marked against the 4.0 release.
>> [1]
>>>>>>
>>>>>>
>>>>>> Things seem to have settled into a phase of working to resolve
>>> issues,
>>>>> with
>>>>>> few new issues added.
>>>>>>
>>>>>>
>>>>>> 2 new tickets opened (that are marked against 4.0)
>>>>>>
>>>>>> 11 tickets closed (including one of the newly opened ones)
>>>>>>
>>>>>> 39 tickets received updates to JIRA of some kind in the last
>> week
>>>>>>
>>>>>>
>>>>>> Cumulative flow over the last couple of weeks shows todo
>> reducing and
>>>>> done
>>>>>> increasing as it should as we continue to close out work for
>> the
>>>> release.
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=14
>>>>>>
>>>>>>
>>>>>> Notables
>>>>>>
>>>>>> - Python 3 support for cqlsh has been committed (thank you all
>> who
>>>>>> persevered on this)
>>>>>>
>>>>>> - Some activity on Windows support - perhaps not dead yet.
>>>>>>
>>>>>> - Lots of movement on documentation
>>>>>>
>>>>>> - Lots of activity on flaky tests.
>>>>>>
>>>>>> - Oldest ticket with a patch award goes to CASSANDRA-2848
>>>>>>
>>>>>>
>>>>>> There are 18 tickets marked as patch available (easy access
>> from the
>>>>>> Dashboard [2], apologies if they're already picked up for
>> review)
>>>>>>
>>>>>>
>>>>>> CASSANDRA-15567 Allow EXTRA_CLASSPATH to work in tarball/source
>>>>>> installations
>>>>>>
>>>>>> CASSANDRA-15553 Preview repair should include sstables from
>> finalized
>>>>>> incremental repair sessions
>>>>>>
>>>>>> CASSANDRA-15550 Fix flaky test
>>>>>> org.apache.cassandra.streaming.StreamTransferTaskTest
>>>>>> testFailSessionDuringTransferShouldNotReleaseReferences
>>>>>>
>>>>>> CASSANDRA-15488/CASSANDRA-15353 Configuration file
>>>>>>
>>>>>> CASSANDRA-15484/CASSANDRA-15353 Read Repair
>>>>>>
>>>>>> CASSANDRA-15482/CASSANDRA-15353 Guarantees
>>>>>>
>>>>>> CASSANDRA-15481/CASSANDRA-15353 Data Modeling
>>>>>>
>>>>>> CASSANDRA-15393/CASSANDRA-15387 Add byte array backed cells
>>>>>>
>>>>>> CASSANDRA-15391/CASSANDRA-15387 Reduce heap footprint of
>> commonly
>>>>> allocated
>>>>>> objects
>>>>>>
>>>>>> CASSANDRA-15367 Memtable memory allocations may deadlock
>>>>>>
>>>>>> CASSANDRA-15308 Fix flakey testAcquireReleaseOutbound -
>>>>>> org.apache.cassandra.net.ConnectionTest
>>>>>>
>>>>>> CASSANDRA-1530 5Fix multi DC nodetool status output
>>>>>>
>>>>>> CASSANDRA-14973 Bring v5 driver out of beta, introduce v6
>> before 4.0
>>>>>> release is cut
>>>>>>
>>>>>> CASSANDRA-14939 fix some operational holes in incremental
>> repair
>>>>>>
>>>>>> CASSANDRA-14904 SSTableloader doesn't understand listening for
>> CQL
>>>>>> connections on multiple ports
>>>>>>
>>>>>> CASSANDRA-14842 SSL connection problems when upgrading to 4.0
>> when
>>>>>> upgrading from 3.0.x
>>>>>>
>>>>>> CASSANDRA-14761 Rename speculative_retry to match
>>>> additional_write_policy
>>>>>>
>>>>>> CASSANDRA-2848 Make the Client API support passing down
>> timeouts
>>>>>>
>>>>>>
>>>>>> *LHF / Failing Tests*: We have 7 unassigned test failures that
>> are
>>> all
>>>>>>
>>>>>> great candidates to pick up and get involved in:
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&quickFilter=1660&quickFilter=1661&quickFilter=1658
>>>>>>
>>>>>>
>>>>>> Thanks again to everybody for all the contributions. It's
>> really
>>> good to
>>>>>> see the open issue count start dropping.
>>>>>>
>>>>>>
>>>>>> Feedback on whether this information is useful and how it can
>> be
>>>> improved
>>>>>> is both welcome and appreciated.
>>>>>>
>>>>>>
>>>>>> Cheers, Jon
>>>>>>
>>>>>>
>>>>>> [1] Unresolved 4.0 tickets
>>>>>>
>>>>>
>>>>
>>>
>> https://issues.apache.org/jira/browse/CASSANDRA-15567?filter=12347782&jql=project%20%3D%20cassandra%20AND%20fixversion%20in%20(4.0%2C%204.0.0%2C%204.0-alpha%2C%204.0-beta)%20AND%20status%20!%3D%20Resolved
>>>>>>
>>>>>> [2] Patch Available
>>>>>>
>>>>>
>>>
>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12334910
>>>>>
>>>>>
>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>>>
>>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>>
>>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: dev-help@cassandra.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Client protocol changes (Was: 20200217 4.0 Status Update)
Posted by Jorge Bay Gondra <jo...@gmail.com>.
Also worth mentioning that, from the driver's perspective, it has to
support a protocol version during the lifetime of the C* version line. For
example, the drivers should drop support for protocol v3 after C* 2.1 goes
EOL, somewhere this year, a protocol that was released back in 2014.
We _could_ establish looser restrictions on whats a breaking change in a
protocol version (needing a version bump), that way the driver can support
a protocol version partially and a protocol version could evolve within
those limits.
Back to the query timeout, a new query flag that can only be set by the
client is not a breaking change for the driver. The driver could ask
whether that feature of the protocol v5 is supported (OPTIONS/SUPPORTED
messages), without having to identify the server version.
On Wed, Feb 19, 2020 at 12:24 AM Benedict Elliott Smith <be...@apache.org>
wrote:
> Behaviours don't have to be switched only with a new protocol version;
> it's possible to support optional feature/modifier flags, the support for
> which is negotiated with a client on connection.
>
> A protocol version change seems reasonable to limit to major releases, but
> a protocol feature seems perfectly reasonable to introduce in a minor, I
> think? Ideally a version change would only be necessary for forced
> deprecation/standardisation of features, behaviour and stream encodings.
>
>
> On 18/02/2020, 21:53, "Jeff Jirsa" <jj...@gmail.com> wrote:
>
> A few notes:
>
> - Protocol changes add work to the rest of the ecosystem. Drivers have
> to
> update, etc.
> - Nobody expects protocol changes in minors, though it's less of a
> concern
> if we don't deprecate out the older version. E.g. if 4.0 launches with
> protocol v4 and protocol v5, and then 4.0.2 adds protocol v6, do we
> deprecate out v4? If yes, you potentially break clients that only
> supported
> v3 and v4 in a minor version upgrade, which is unexpected. If not, how
> many
> protocol versions are you willing to support at any given time?
> - Having protocol changes introduces risk. Paging behavior across
> protocol
> versions is the site of a number of different bugs recently.
>
>
> On Tue, Feb 18, 2020 at 1:46 PM Tolbert, Andrew <x...@andrewtolbert.com>
> wrote:
>
> > I don't know the technical answer, but I suspect two motivations for
> > doing new protocol versions in major releases could include:
> >
> > * protocol changes can be tied to feature changes that typically come
> > in a major release.
> > * protocol changes should be as infrequent as major releases. Each
> > new protocol version is another thing in the test matrix that needs
> to
> > be tested.
> >
> > That last point can make it hard to get new changes in. If something
> > doesn't make the upcoming protocol version, it might be years before
> > another one, but I also think it's worth it to do this infrequently
> as
> > it makes maintaining client and server code easier if there are less
> > protocol versions to worry about.
> >
> > On the client-side, libraries themselves should be avoiding making
> > Cassandra version checks when detecting capabilities. There are a
> few
> > exceptions, such as system table parsing for schema & peers,
> > but those aren't related to the protocol.
> >
> > Thanks,
> > Andy
> >
> >
> >
> >
> >
> > On Tue, Feb 18, 2020 at 1:22 PM Nate McCall <zz...@gmail.com>
> wrote:
> > >
> > > [Moving to new message thread]
> > >
> > > Thanks for bringing this up, Jordan.
> > >
> > > IIRC, this was more a convention than a technical reason. Though I
> could
> > be
> > > completely misremembering this.
> > >
> > > ---------- Forwarded message ---------
> > > From: Jordan West <jo...@gmail.com>
> > > Date: Wed, Feb 19, 2020 at 10:13 AM
> > > Subject: Re: 20200217 4.0 Status Update
> > > To: <de...@cassandra.apache.org>
> > >
> > >
> > > On Mon, Feb 17, 2020 at 12:52 PM Jeff Jirsa <jj...@gmail.com>
> wrote:
> > >
> > > >
> > > > beyond the client proto change being painful for anything other
> than
> > major
> > > > releases
> > > >
> > > >
> > > This came up during the community meeting today and I wanted to
> bring a
> > > question about it to the list: could someone who is *very*
> familiar with
> > > the client proto share w/ the list why changing the proto in
> anything
> > other
> > > than a major release is so difficult? I hear this a lot and it
> seems to
> > be
> > > fact. So that all of us don't have to go read the code, a brief
> summary
> > > would be super helpful. Or if there is a ticket that already
> covers this
> > > even better! I'd also be curious if there have ever been any
> thoughts to
> > > address it as it seems to be a consistent hurdle during the
> release cycle
> > > and one that tends to further increase scope.
> > >
> > > Thanks,
> > > Jordan
> > >
> > > >
> > > >
> > > > > On Feb 17, 2020, at 12:43 PM, Jon Meredith <
> jmeredithco@gmail.com>
> > > > wrote:
> > > > >
> > > > > My turn to give an update on 4.0 status. The 4.0 board
> created by
> > Josh
> > > > can
> > > > > be found at
> > > > >
> > > > >
> > > > >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355.
> > > > >
> > > > >
> > > > > We have 94 unresolved tickets marked against the 4.0 release.
> [1]
> > > > >
> > > > >
> > > > > Things seem to have settled into a phase of working to resolve
> > issues,
> > > > with
> > > > > few new issues added.
> > > > >
> > > > >
> > > > > 2 new tickets opened (that are marked against 4.0)
> > > > >
> > > > > 11 tickets closed (including one of the newly opened ones)
> > > > >
> > > > > 39 tickets received updates to JIRA of some kind in the last
> week
> > > > >
> > > > >
> > > > > Cumulative flow over the last couple of weeks shows todo
> reducing and
> > > > done
> > > > > increasing as it should as we continue to close out work for
> the
> > > release.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=14
> > > > >
> > > > >
> > > > > Notables
> > > > >
> > > > > - Python 3 support for cqlsh has been committed (thank you all
> who
> > > > > persevered on this)
> > > > >
> > > > > - Some activity on Windows support - perhaps not dead yet.
> > > > >
> > > > > - Lots of movement on documentation
> > > > >
> > > > > - Lots of activity on flaky tests.
> > > > >
> > > > > - Oldest ticket with a patch award goes to CASSANDRA-2848
> > > > >
> > > > >
> > > > > There are 18 tickets marked as patch available (easy access
> from the
> > > > > Dashboard [2], apologies if they're already picked up for
> review)
> > > > >
> > > > >
> > > > > CASSANDRA-15567 Allow EXTRA_CLASSPATH to work in tarball/source
> > > > > installations
> > > > >
> > > > > CASSANDRA-15553 Preview repair should include sstables from
> finalized
> > > > > incremental repair sessions
> > > > >
> > > > > CASSANDRA-15550 Fix flaky test
> > > > > org.apache.cassandra.streaming.StreamTransferTaskTest
> > > > > testFailSessionDuringTransferShouldNotReleaseReferences
> > > > >
> > > > > CASSANDRA-15488/CASSANDRA-15353 Configuration file
> > > > >
> > > > > CASSANDRA-15484/CASSANDRA-15353 Read Repair
> > > > >
> > > > > CASSANDRA-15482/CASSANDRA-15353 Guarantees
> > > > >
> > > > > CASSANDRA-15481/CASSANDRA-15353 Data Modeling
> > > > >
> > > > > CASSANDRA-15393/CASSANDRA-15387 Add byte array backed cells
> > > > >
> > > > > CASSANDRA-15391/CASSANDRA-15387 Reduce heap footprint of
> commonly
> > > > allocated
> > > > > objects
> > > > >
> > > > > CASSANDRA-15367 Memtable memory allocations may deadlock
> > > > >
> > > > > CASSANDRA-15308 Fix flakey testAcquireReleaseOutbound -
> > > > > org.apache.cassandra.net.ConnectionTest
> > > > >
> > > > > CASSANDRA-1530 5Fix multi DC nodetool status output
> > > > >
> > > > > CASSANDRA-14973 Bring v5 driver out of beta, introduce v6
> before 4.0
> > > > > release is cut
> > > > >
> > > > > CASSANDRA-14939 fix some operational holes in incremental
> repair
> > > > >
> > > > > CASSANDRA-14904 SSTableloader doesn't understand listening for
> CQL
> > > > > connections on multiple ports
> > > > >
> > > > > CASSANDRA-14842 SSL connection problems when upgrading to 4.0
> when
> > > > > upgrading from 3.0.x
> > > > >
> > > > > CASSANDRA-14761 Rename speculative_retry to match
> > > additional_write_policy
> > > > >
> > > > > CASSANDRA-2848 Make the Client API support passing down
> timeouts
> > > > >
> > > > >
> > > > > *LHF / Failing Tests*: We have 7 unassigned test failures that
> are
> > all
> > > > >
> > > > > great candidates to pick up and get involved in:
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&quickFilter=1660&quickFilter=1661&quickFilter=1658
> > > > >
> > > > >
> > > > > Thanks again to everybody for all the contributions. It's
> really
> > good to
> > > > > see the open issue count start dropping.
> > > > >
> > > > >
> > > > > Feedback on whether this information is useful and how it can
> be
> > > improved
> > > > > is both welcome and appreciated.
> > > > >
> > > > >
> > > > > Cheers, Jon
> > > > >
> > > > >
> > > > > [1] Unresolved 4.0 tickets
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/CASSANDRA-15567?filter=12347782&jql=project%20%3D%20cassandra%20AND%20fixversion%20in%20(4.0%2C%204.0.0%2C%204.0-alpha%2C%204.0-beta)%20AND%20status%20!%3D%20Resolved
> > > > >
> > > > > [2] Patch Available
> > > > >
> > > >
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12334910
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] Client protocol changes (Was: 20200217 4.0 Status
Update)
Posted by Benedict Elliott Smith <be...@apache.org>.
Behaviours don't have to be switched only with a new protocol version; it's possible to support optional feature/modifier flags, the support for which is negotiated with a client on connection.
A protocol version change seems reasonable to limit to major releases, but a protocol feature seems perfectly reasonable to introduce in a minor, I think? Ideally a version change would only be necessary for forced deprecation/standardisation of features, behaviour and stream encodings.
On 18/02/2020, 21:53, "Jeff Jirsa" <jj...@gmail.com> wrote:
A few notes:
- Protocol changes add work to the rest of the ecosystem. Drivers have to
update, etc.
- Nobody expects protocol changes in minors, though it's less of a concern
if we don't deprecate out the older version. E.g. if 4.0 launches with
protocol v4 and protocol v5, and then 4.0.2 adds protocol v6, do we
deprecate out v4? If yes, you potentially break clients that only supported
v3 and v4 in a minor version upgrade, which is unexpected. If not, how many
protocol versions are you willing to support at any given time?
- Having protocol changes introduces risk. Paging behavior across protocol
versions is the site of a number of different bugs recently.
On Tue, Feb 18, 2020 at 1:46 PM Tolbert, Andrew <x...@andrewtolbert.com> wrote:
> I don't know the technical answer, but I suspect two motivations for
> doing new protocol versions in major releases could include:
>
> * protocol changes can be tied to feature changes that typically come
> in a major release.
> * protocol changes should be as infrequent as major releases. Each
> new protocol version is another thing in the test matrix that needs to
> be tested.
>
> That last point can make it hard to get new changes in. If something
> doesn't make the upcoming protocol version, it might be years before
> another one, but I also think it's worth it to do this infrequently as
> it makes maintaining client and server code easier if there are less
> protocol versions to worry about.
>
> On the client-side, libraries themselves should be avoiding making
> Cassandra version checks when detecting capabilities. There are a few
> exceptions, such as system table parsing for schema & peers,
> but those aren't related to the protocol.
>
> Thanks,
> Andy
>
>
>
>
>
> On Tue, Feb 18, 2020 at 1:22 PM Nate McCall <zz...@gmail.com> wrote:
> >
> > [Moving to new message thread]
> >
> > Thanks for bringing this up, Jordan.
> >
> > IIRC, this was more a convention than a technical reason. Though I could
> be
> > completely misremembering this.
> >
> > ---------- Forwarded message ---------
> > From: Jordan West <jo...@gmail.com>
> > Date: Wed, Feb 19, 2020 at 10:13 AM
> > Subject: Re: 20200217 4.0 Status Update
> > To: <de...@cassandra.apache.org>
> >
> >
> > On Mon, Feb 17, 2020 at 12:52 PM Jeff Jirsa <jj...@gmail.com> wrote:
> >
> > >
> > > beyond the client proto change being painful for anything other than
> major
> > > releases
> > >
> > >
> > This came up during the community meeting today and I wanted to bring a
> > question about it to the list: could someone who is *very* familiar with
> > the client proto share w/ the list why changing the proto in anything
> other
> > than a major release is so difficult? I hear this a lot and it seems to
> be
> > fact. So that all of us don't have to go read the code, a brief summary
> > would be super helpful. Or if there is a ticket that already covers this
> > even better! I'd also be curious if there have ever been any thoughts to
> > address it as it seems to be a consistent hurdle during the release cycle
> > and one that tends to further increase scope.
> >
> > Thanks,
> > Jordan
> >
> > >
> > >
> > > > On Feb 17, 2020, at 12:43 PM, Jon Meredith <jm...@gmail.com>
> > > wrote:
> > > >
> > > > My turn to give an update on 4.0 status. The 4.0 board created by
> Josh
> > > can
> > > > be found at
> > > >
> > > >
> > > > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355.
> > > >
> > > >
> > > > We have 94 unresolved tickets marked against the 4.0 release. [1]
> > > >
> > > >
> > > > Things seem to have settled into a phase of working to resolve
> issues,
> > > with
> > > > few new issues added.
> > > >
> > > >
> > > > 2 new tickets opened (that are marked against 4.0)
> > > >
> > > > 11 tickets closed (including one of the newly opened ones)
> > > >
> > > > 39 tickets received updates to JIRA of some kind in the last week
> > > >
> > > >
> > > > Cumulative flow over the last couple of weeks shows todo reducing and
> > > done
> > > > increasing as it should as we continue to close out work for the
> > release.
> > > >
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=14
> > > >
> > > >
> > > > Notables
> > > >
> > > > - Python 3 support for cqlsh has been committed (thank you all who
> > > > persevered on this)
> > > >
> > > > - Some activity on Windows support - perhaps not dead yet.
> > > >
> > > > - Lots of movement on documentation
> > > >
> > > > - Lots of activity on flaky tests.
> > > >
> > > > - Oldest ticket with a patch award goes to CASSANDRA-2848
> > > >
> > > >
> > > > There are 18 tickets marked as patch available (easy access from the
> > > > Dashboard [2], apologies if they're already picked up for review)
> > > >
> > > >
> > > > CASSANDRA-15567 Allow EXTRA_CLASSPATH to work in tarball/source
> > > > installations
> > > >
> > > > CASSANDRA-15553 Preview repair should include sstables from finalized
> > > > incremental repair sessions
> > > >
> > > > CASSANDRA-15550 Fix flaky test
> > > > org.apache.cassandra.streaming.StreamTransferTaskTest
> > > > testFailSessionDuringTransferShouldNotReleaseReferences
> > > >
> > > > CASSANDRA-15488/CASSANDRA-15353 Configuration file
> > > >
> > > > CASSANDRA-15484/CASSANDRA-15353 Read Repair
> > > >
> > > > CASSANDRA-15482/CASSANDRA-15353 Guarantees
> > > >
> > > > CASSANDRA-15481/CASSANDRA-15353 Data Modeling
> > > >
> > > > CASSANDRA-15393/CASSANDRA-15387 Add byte array backed cells
> > > >
> > > > CASSANDRA-15391/CASSANDRA-15387 Reduce heap footprint of commonly
> > > allocated
> > > > objects
> > > >
> > > > CASSANDRA-15367 Memtable memory allocations may deadlock
> > > >
> > > > CASSANDRA-15308 Fix flakey testAcquireReleaseOutbound -
> > > > org.apache.cassandra.net.ConnectionTest
> > > >
> > > > CASSANDRA-1530 5Fix multi DC nodetool status output
> > > >
> > > > CASSANDRA-14973 Bring v5 driver out of beta, introduce v6 before 4.0
> > > > release is cut
> > > >
> > > > CASSANDRA-14939 fix some operational holes in incremental repair
> > > >
> > > > CASSANDRA-14904 SSTableloader doesn't understand listening for CQL
> > > > connections on multiple ports
> > > >
> > > > CASSANDRA-14842 SSL connection problems when upgrading to 4.0 when
> > > > upgrading from 3.0.x
> > > >
> > > > CASSANDRA-14761 Rename speculative_retry to match
> > additional_write_policy
> > > >
> > > > CASSANDRA-2848 Make the Client API support passing down timeouts
> > > >
> > > >
> > > > *LHF / Failing Tests*: We have 7 unassigned test failures that are
> all
> > > >
> > > > great candidates to pick up and get involved in:
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&quickFilter=1660&quickFilter=1661&quickFilter=1658
> > > >
> > > >
> > > > Thanks again to everybody for all the contributions. It's really
> good to
> > > > see the open issue count start dropping.
> > > >
> > > >
> > > > Feedback on whether this information is useful and how it can be
> > improved
> > > > is both welcome and appreciated.
> > > >
> > > >
> > > > Cheers, Jon
> > > >
> > > >
> > > > [1] Unresolved 4.0 tickets
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/CASSANDRA-15567?filter=12347782&jql=project%20%3D%20cassandra%20AND%20fixversion%20in%20(4.0%2C%204.0.0%2C%204.0-alpha%2C%204.0-beta)%20AND%20status%20!%3D%20Resolved
> > > >
> > > > [2] Patch Available
> > > >
> > >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12334910
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Client protocol changes (Was: 20200217 4.0 Status Update)
Posted by Jeff Jirsa <jj...@gmail.com>.
A few notes:
- Protocol changes add work to the rest of the ecosystem. Drivers have to
update, etc.
- Nobody expects protocol changes in minors, though it's less of a concern
if we don't deprecate out the older version. E.g. if 4.0 launches with
protocol v4 and protocol v5, and then 4.0.2 adds protocol v6, do we
deprecate out v4? If yes, you potentially break clients that only supported
v3 and v4 in a minor version upgrade, which is unexpected. If not, how many
protocol versions are you willing to support at any given time?
- Having protocol changes introduces risk. Paging behavior across protocol
versions is the site of a number of different bugs recently.
On Tue, Feb 18, 2020 at 1:46 PM Tolbert, Andrew <x...@andrewtolbert.com> wrote:
> I don't know the technical answer, but I suspect two motivations for
> doing new protocol versions in major releases could include:
>
> * protocol changes can be tied to feature changes that typically come
> in a major release.
> * protocol changes should be as infrequent as major releases. Each
> new protocol version is another thing in the test matrix that needs to
> be tested.
>
> That last point can make it hard to get new changes in. If something
> doesn't make the upcoming protocol version, it might be years before
> another one, but I also think it's worth it to do this infrequently as
> it makes maintaining client and server code easier if there are less
> protocol versions to worry about.
>
> On the client-side, libraries themselves should be avoiding making
> Cassandra version checks when detecting capabilities. There are a few
> exceptions, such as system table parsing for schema & peers,
> but those aren't related to the protocol.
>
> Thanks,
> Andy
>
>
>
>
>
> On Tue, Feb 18, 2020 at 1:22 PM Nate McCall <zz...@gmail.com> wrote:
> >
> > [Moving to new message thread]
> >
> > Thanks for bringing this up, Jordan.
> >
> > IIRC, this was more a convention than a technical reason. Though I could
> be
> > completely misremembering this.
> >
> > ---------- Forwarded message ---------
> > From: Jordan West <jo...@gmail.com>
> > Date: Wed, Feb 19, 2020 at 10:13 AM
> > Subject: Re: 20200217 4.0 Status Update
> > To: <de...@cassandra.apache.org>
> >
> >
> > On Mon, Feb 17, 2020 at 12:52 PM Jeff Jirsa <jj...@gmail.com> wrote:
> >
> > >
> > > beyond the client proto change being painful for anything other than
> major
> > > releases
> > >
> > >
> > This came up during the community meeting today and I wanted to bring a
> > question about it to the list: could someone who is *very* familiar with
> > the client proto share w/ the list why changing the proto in anything
> other
> > than a major release is so difficult? I hear this a lot and it seems to
> be
> > fact. So that all of us don't have to go read the code, a brief summary
> > would be super helpful. Or if there is a ticket that already covers this
> > even better! I'd also be curious if there have ever been any thoughts to
> > address it as it seems to be a consistent hurdle during the release cycle
> > and one that tends to further increase scope.
> >
> > Thanks,
> > Jordan
> >
> > >
> > >
> > > > On Feb 17, 2020, at 12:43 PM, Jon Meredith <jm...@gmail.com>
> > > wrote:
> > > >
> > > > My turn to give an update on 4.0 status. The 4.0 board created by
> Josh
> > > can
> > > > be found at
> > > >
> > > >
> > > > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355.
> > > >
> > > >
> > > > We have 94 unresolved tickets marked against the 4.0 release. [1]
> > > >
> > > >
> > > > Things seem to have settled into a phase of working to resolve
> issues,
> > > with
> > > > few new issues added.
> > > >
> > > >
> > > > 2 new tickets opened (that are marked against 4.0)
> > > >
> > > > 11 tickets closed (including one of the newly opened ones)
> > > >
> > > > 39 tickets received updates to JIRA of some kind in the last week
> > > >
> > > >
> > > > Cumulative flow over the last couple of weeks shows todo reducing and
> > > done
> > > > increasing as it should as we continue to close out work for the
> > release.
> > > >
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=14
> > > >
> > > >
> > > > Notables
> > > >
> > > > - Python 3 support for cqlsh has been committed (thank you all who
> > > > persevered on this)
> > > >
> > > > - Some activity on Windows support - perhaps not dead yet.
> > > >
> > > > - Lots of movement on documentation
> > > >
> > > > - Lots of activity on flaky tests.
> > > >
> > > > - Oldest ticket with a patch award goes to CASSANDRA-2848
> > > >
> > > >
> > > > There are 18 tickets marked as patch available (easy access from the
> > > > Dashboard [2], apologies if they're already picked up for review)
> > > >
> > > >
> > > > CASSANDRA-15567 Allow EXTRA_CLASSPATH to work in tarball/source
> > > > installations
> > > >
> > > > CASSANDRA-15553 Preview repair should include sstables from finalized
> > > > incremental repair sessions
> > > >
> > > > CASSANDRA-15550 Fix flaky test
> > > > org.apache.cassandra.streaming.StreamTransferTaskTest
> > > > testFailSessionDuringTransferShouldNotReleaseReferences
> > > >
> > > > CASSANDRA-15488/CASSANDRA-15353 Configuration file
> > > >
> > > > CASSANDRA-15484/CASSANDRA-15353 Read Repair
> > > >
> > > > CASSANDRA-15482/CASSANDRA-15353 Guarantees
> > > >
> > > > CASSANDRA-15481/CASSANDRA-15353 Data Modeling
> > > >
> > > > CASSANDRA-15393/CASSANDRA-15387 Add byte array backed cells
> > > >
> > > > CASSANDRA-15391/CASSANDRA-15387 Reduce heap footprint of commonly
> > > allocated
> > > > objects
> > > >
> > > > CASSANDRA-15367 Memtable memory allocations may deadlock
> > > >
> > > > CASSANDRA-15308 Fix flakey testAcquireReleaseOutbound -
> > > > org.apache.cassandra.net.ConnectionTest
> > > >
> > > > CASSANDRA-1530 5Fix multi DC nodetool status output
> > > >
> > > > CASSANDRA-14973 Bring v5 driver out of beta, introduce v6 before 4.0
> > > > release is cut
> > > >
> > > > CASSANDRA-14939 fix some operational holes in incremental repair
> > > >
> > > > CASSANDRA-14904 SSTableloader doesn't understand listening for CQL
> > > > connections on multiple ports
> > > >
> > > > CASSANDRA-14842 SSL connection problems when upgrading to 4.0 when
> > > > upgrading from 3.0.x
> > > >
> > > > CASSANDRA-14761 Rename speculative_retry to match
> > additional_write_policy
> > > >
> > > > CASSANDRA-2848 Make the Client API support passing down timeouts
> > > >
> > > >
> > > > *LHF / Failing Tests*: We have 7 unassigned test failures that are
> all
> > > >
> > > > great candidates to pick up and get involved in:
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&quickFilter=1660&quickFilter=1661&quickFilter=1658
> > > >
> > > >
> > > > Thanks again to everybody for all the contributions. It's really
> good to
> > > > see the open issue count start dropping.
> > > >
> > > >
> > > > Feedback on whether this information is useful and how it can be
> > improved
> > > > is both welcome and appreciated.
> > > >
> > > >
> > > > Cheers, Jon
> > > >
> > > >
> > > > [1] Unresolved 4.0 tickets
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/CASSANDRA-15567?filter=12347782&jql=project%20%3D%20cassandra%20AND%20fixversion%20in%20(4.0%2C%204.0.0%2C%204.0-alpha%2C%204.0-beta)%20AND%20status%20!%3D%20Resolved
> > > >
> > > > [2] Patch Available
> > > >
> > >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12334910
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > > For additional commands, e-mail: dev-help@cassandra.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] Client protocol changes (Was: 20200217 4.0 Status Update)
Posted by "Tolbert, Andrew" <x...@andrewtolbert.com>.
I don't know the technical answer, but I suspect two motivations for
doing new protocol versions in major releases could include:
* protocol changes can be tied to feature changes that typically come
in a major release.
* protocol changes should be as infrequent as major releases. Each
new protocol version is another thing in the test matrix that needs to
be tested.
That last point can make it hard to get new changes in. If something
doesn't make the upcoming protocol version, it might be years before
another one, but I also think it's worth it to do this infrequently as
it makes maintaining client and server code easier if there are less
protocol versions to worry about.
On the client-side, libraries themselves should be avoiding making
Cassandra version checks when detecting capabilities. There are a few
exceptions, such as system table parsing for schema & peers,
but those aren't related to the protocol.
Thanks,
Andy
On Tue, Feb 18, 2020 at 1:22 PM Nate McCall <zz...@gmail.com> wrote:
>
> [Moving to new message thread]
>
> Thanks for bringing this up, Jordan.
>
> IIRC, this was more a convention than a technical reason. Though I could be
> completely misremembering this.
>
> ---------- Forwarded message ---------
> From: Jordan West <jo...@gmail.com>
> Date: Wed, Feb 19, 2020 at 10:13 AM
> Subject: Re: 20200217 4.0 Status Update
> To: <de...@cassandra.apache.org>
>
>
> On Mon, Feb 17, 2020 at 12:52 PM Jeff Jirsa <jj...@gmail.com> wrote:
>
> >
> > beyond the client proto change being painful for anything other than major
> > releases
> >
> >
> This came up during the community meeting today and I wanted to bring a
> question about it to the list: could someone who is *very* familiar with
> the client proto share w/ the list why changing the proto in anything other
> than a major release is so difficult? I hear this a lot and it seems to be
> fact. So that all of us don't have to go read the code, a brief summary
> would be super helpful. Or if there is a ticket that already covers this
> even better! I'd also be curious if there have ever been any thoughts to
> address it as it seems to be a consistent hurdle during the release cycle
> and one that tends to further increase scope.
>
> Thanks,
> Jordan
>
> >
> >
> > > On Feb 17, 2020, at 12:43 PM, Jon Meredith <jm...@gmail.com>
> > wrote:
> > >
> > > My turn to give an update on 4.0 status. The 4.0 board created by Josh
> > can
> > > be found at
> > >
> > >
> > > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355.
> > >
> > >
> > > We have 94 unresolved tickets marked against the 4.0 release. [1]
> > >
> > >
> > > Things seem to have settled into a phase of working to resolve issues,
> > with
> > > few new issues added.
> > >
> > >
> > > 2 new tickets opened (that are marked against 4.0)
> > >
> > > 11 tickets closed (including one of the newly opened ones)
> > >
> > > 39 tickets received updates to JIRA of some kind in the last week
> > >
> > >
> > > Cumulative flow over the last couple of weeks shows todo reducing and
> > done
> > > increasing as it should as we continue to close out work for the
> release.
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&view=reporting&chart=cumulativeFlowDiagram&swimlane=939&swimlane=936&swimlane=931&column=1505&column=1506&column=1514&column=1509&column=1512&column=1507&days=14
> > >
> > >
> > > Notables
> > >
> > > - Python 3 support for cqlsh has been committed (thank you all who
> > > persevered on this)
> > >
> > > - Some activity on Windows support - perhaps not dead yet.
> > >
> > > - Lots of movement on documentation
> > >
> > > - Lots of activity on flaky tests.
> > >
> > > - Oldest ticket with a patch award goes to CASSANDRA-2848
> > >
> > >
> > > There are 18 tickets marked as patch available (easy access from the
> > > Dashboard [2], apologies if they're already picked up for review)
> > >
> > >
> > > CASSANDRA-15567 Allow EXTRA_CLASSPATH to work in tarball/source
> > > installations
> > >
> > > CASSANDRA-15553 Preview repair should include sstables from finalized
> > > incremental repair sessions
> > >
> > > CASSANDRA-15550 Fix flaky test
> > > org.apache.cassandra.streaming.StreamTransferTaskTest
> > > testFailSessionDuringTransferShouldNotReleaseReferences
> > >
> > > CASSANDRA-15488/CASSANDRA-15353 Configuration file
> > >
> > > CASSANDRA-15484/CASSANDRA-15353 Read Repair
> > >
> > > CASSANDRA-15482/CASSANDRA-15353 Guarantees
> > >
> > > CASSANDRA-15481/CASSANDRA-15353 Data Modeling
> > >
> > > CASSANDRA-15393/CASSANDRA-15387 Add byte array backed cells
> > >
> > > CASSANDRA-15391/CASSANDRA-15387 Reduce heap footprint of commonly
> > allocated
> > > objects
> > >
> > > CASSANDRA-15367 Memtable memory allocations may deadlock
> > >
> > > CASSANDRA-15308 Fix flakey testAcquireReleaseOutbound -
> > > org.apache.cassandra.net.ConnectionTest
> > >
> > > CASSANDRA-1530 5Fix multi DC nodetool status output
> > >
> > > CASSANDRA-14973 Bring v5 driver out of beta, introduce v6 before 4.0
> > > release is cut
> > >
> > > CASSANDRA-14939 fix some operational holes in incremental repair
> > >
> > > CASSANDRA-14904 SSTableloader doesn't understand listening for CQL
> > > connections on multiple ports
> > >
> > > CASSANDRA-14842 SSL connection problems when upgrading to 4.0 when
> > > upgrading from 3.0.x
> > >
> > > CASSANDRA-14761 Rename speculative_retry to match
> additional_write_policy
> > >
> > > CASSANDRA-2848 Make the Client API support passing down timeouts
> > >
> > >
> > > *LHF / Failing Tests*: We have 7 unassigned test failures that are all
> > >
> > > great candidates to pick up and get involved in:
> > >
> > >
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&projectKey=CASSANDRA&quickFilter=1660&quickFilter=1661&quickFilter=1658
> > >
> > >
> > > Thanks again to everybody for all the contributions. It's really good to
> > > see the open issue count start dropping.
> > >
> > >
> > > Feedback on whether this information is useful and how it can be
> improved
> > > is both welcome and appreciated.
> > >
> > >
> > > Cheers, Jon
> > >
> > >
> > > [1] Unresolved 4.0 tickets
> > >
> >
> https://issues.apache.org/jira/browse/CASSANDRA-15567?filter=12347782&jql=project%20%3D%20cassandra%20AND%20fixversion%20in%20(4.0%2C%204.0.0%2C%204.0-alpha%2C%204.0-beta)%20AND%20status%20!%3D%20Resolved
> > >
> > > [2] Patch Available
> > >
> > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12334910
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org