You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Oleksandr Petrov <ol...@gmail.com> on 2020/10/22 07:19:41 UTC
Minimal 4.0 COMPACT STORAGE backport
Since this is an important subject, I thought it also makes sense to start
a mailing list thread.
You may know that in 4.0 there was a plan to drop compact storage and
related code. However, there are several behavioural changes related to
compact storage, and difference in visible behaviour between "normal" and
compact tables are larger than most of us have anticipated: we first
thought there’ll be only “appearing column” in dense case, but there’s
implicit nulls in clusterings thing, and row vs column deletion now, TTL,
and more.
Some of the recent issues on the subject are: CASSANDRA-16048
<https://issues.apache.org/jira/browse/CASSANDRA-16048>, which allows to
ignore these differences. The other one was an attempt to improve the user
experience of anyone still using compact storage: CASSANDRA-15811
<https://issues.apache.org/jira/browse/CASSANDRA-15811>.
Easily reproducible differences are:
(1) hidden columns show up, which breaks SELECT * queries
(2) DELETE v and UPDATE v WITH TTL would result into row removals in
non-dense compact tables (CASSANDRA-16069
<https://issues.apache.org/jira/browse/CASSANDRA-16069>)
(3) INSERT allows skipping clusterings, which are filled with nulls by
default.
Some of these are tricky to support, as 15811 has shown. Anyone who might
want to upgrade to 4.0 while still using compact storage might be affected
by being forced into one of these behaviours.
Possible solutions are to document these behaviours, or to bring back a
minimal set of COMPACT STORAGE and keep supporting these in 4.0
It looks like it is possible to leave some of the functionality related to
DENSE flag and allow it to be present in 4.0, but only for these three (and
potential related, however not directly visible) cases.
You can find more details on the subject here:
https://issues.apache.org/jira/browse/CASSANDRA-16217
Thank you,
-- Alex
Re: Minimal 4.0 COMPACT STORAGE backport
Posted by Caleb Rackliffe <ca...@gmail.com>.
+1 to this, and my immediate concern is the possibility that the patch I'm
working on for CASSANDRA-16226
<https://issues.apache.org/jira/browse/CASSANDRA-16226> will actually
require information about whether a table was *created* with COMPACT
STORAGE.
On Thu, Oct 22, 2020 at 2:41 PM David Capwell <dc...@gmail.com> wrote:
> Thanks for the reply.
>
> Talked with Alex in Slack and I was under the impression deleted rows were
> handled differently on read.
>
> collect ideas about migration, cutoff date and final deprecation in a
> > separate thread or a document.
>
>
> +1
>
> I am good bringing a subset back to ease upgrades to 4.0
>
> On Thu, Oct 22, 2020 at 11:42 AM Oleksandr Petrov <
> oleksandr.petrov@gmail.com> wrote:
>
> > These are some great (albeit somewhat hard) questions!
> >
> > > * when you drop compact storage and have some SSTables stored in
> > compact storage and some SSTables stored normally, how do we present data
> > correctly?
> >
> > There is no difference in SSTable format between normal and compact
> > storage, the difference is only in how Table Metadata is represented. In
> > other words, in code.
> >
> > * does upgradesstables and/or compaction migrate off the compact
> > storage format into the normal format?
> >
> > I think above answers this: the only way to migrate off compact storage
> is
> > to change table metadata, and accept the differences mentioned above.
> >
> > * how do we know it is safe to remove?
> >
> > Maybe if we're explicit about it and say that, for example, by version X
> > compact storage will not be supported at all, and document all the
> > pitfalls, people will migrate their applications, one at a time. There's
> > been a few suggestions about how to mitigate, including client options in
> > 10857, and introducing "hiding" columns in 15811, and many more that were
> > discussed but not implemented.
> >
> > Hope this won't discourage further discussions, but I suggest we keep
> this
> > thread focused on pros/cons of a suggested intermediate solution (bring
> > back a minimal subset of CS), and collect ideas about migration, cutoff
> > date and final deprecation in a separate thread or a document. I
> personally
> > do not have a full answer to "when it is safe to remove", and it seems
> like
> > the best we can do is to create a clear procedure.
> >
> > On Thu, Oct 22, 2020 at 6:38 PM Ekaterina Dimitrova <
> e.dimitrova@gmail.com
> > >
> > wrote:
> >
> > > Definitely that is not off the table.
> > > Also, we talked this morning for a plan for additional testing to be
> > > created as part of CASSANDRA-15588
> > >
> > > On Thu, 22 Oct 2020 at 12:33, David Capwell <dc...@gmail.com>
> wrote:
> > >
> > > > I am in favor of bringing it back, but I do feel we should also plan
> > how
> > > to
> > > > get it removed as well.
> > > >
> > > > Few examples, would love to see fleshed out
> > > >
> > > > * when you drop compact storage and have some SSTables stored in
> > compact
> > > > storage and some SSTables stored normally, how do we present data
> > > > correctly?
> > > > * does upgradesstables and/or compaction migrate off the compact
> > storage
> > > > format into the normal format?
> > > > * how do we know it is safe to remove?
> > > >
> > > >
> > > >
> > > > On Thu, Oct 22, 2020 at 6:55 AM Ekaterina Dimitrova <
> > > e.dimitrova@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Alex,
> > > > > Thanks for bringing this up.
> > > > > I am with you for returning part of the code back and considering
> > this
> > > > as a
> > > > > bug.
> > > > > I truly believe it is too late in the release to document changed
> > > > behavior.
> > > > > I think this contradicts with the project’s promise for no breaking
> > > > > changes. This should have been documented in alpha.
> > > > > Best regards,
> > > > > Ekaterina
> > > > >
> > > > > On Thu, 22 Oct 2020 at 3:20, Oleksandr Petrov <
> > > > oleksandr.petrov@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Since this is an important subject, I thought it also makes sense
> > to
> > > > > start
> > > > > > a mailing list thread.
> > > > > >
> > > > > > You may know that in 4.0 there was a plan to drop compact storage
> > and
> > > > > > related code. However, there are several behavioural changes
> > related
> > > to
> > > > > > compact storage, and difference in visible behaviour between
> > "normal"
> > > > and
> > > > > > compact tables are larger than most of us have anticipated: we
> > first
> > > > > > thought there’ll be only “appearing column” in dense case, but
> > > there’s
> > > > > > implicit nulls in clusterings thing, and row vs column deletion
> > now,
> > > > TTL,
> > > > > > and more.
> > > > > >
> > > > > > Some of the recent issues on the subject are: CASSANDRA-16048
> > > > > > <https://issues.apache.org/jira/browse/CASSANDRA-16048>, which
> > > allows
> > > > to
> > > > > > ignore these differences. The other one was an attempt to improve
> > the
> > > > > user
> > > > > > experience of anyone still using compact storage: CASSANDRA-15811
> > > > > > <https://issues.apache.org/jira/browse/CASSANDRA-15811>.
> > > > > >
> > > > > > Easily reproducible differences are:
> > > > > >
> > > > > > (1) hidden columns show up, which breaks SELECT * queries
> > > > > > (2) DELETE v and UPDATE v WITH TTL would result into row removals
> > in
> > > > > > non-dense compact tables (CASSANDRA-16069
> > > > > > <https://issues.apache.org/jira/browse/CASSANDRA-16069>)
> > > > > > (3) INSERT allows skipping clusterings, which are filled with
> nulls
> > > by
> > > > > > default.
> > > > > >
> > > > > > Some of these are tricky to support, as 15811 has shown. Anyone
> who
> > > > might
> > > > > > want to upgrade to 4.0 while still using compact storage might be
> > > > > affected
> > > > > > by being forced into one of these behaviours.
> > > > > >
> > > > > > Possible solutions are to document these behaviours, or to bring
> > > back a
> > > > > > minimal set of COMPACT STORAGE and keep supporting these in 4.0
> > > > > >
> > > > > > It looks like it is possible to leave some of the functionality
> > > related
> > > > > to
> > > > > > DENSE flag and allow it to be present in 4.0, but only for these
> > > three
> > > > > (and
> > > > > > potential related, however not directly visible) cases.
> > > > > >
> > > > > > You can find more details on the subject here:
> > > > > > https://issues.apache.org/jira/browse/CASSANDRA-16217
> > > > > >
> > > > > > Thank you,
> > > > > >
> > > > > > -- Alex
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> > --
> > alex p
> >
>
Re: Minimal 4.0 COMPACT STORAGE backport
Posted by David Capwell <dc...@gmail.com>.
Thanks for the reply.
Talked with Alex in Slack and I was under the impression deleted rows were
handled differently on read.
collect ideas about migration, cutoff date and final deprecation in a
> separate thread or a document.
+1
I am good bringing a subset back to ease upgrades to 4.0
On Thu, Oct 22, 2020 at 11:42 AM Oleksandr Petrov <
oleksandr.petrov@gmail.com> wrote:
> These are some great (albeit somewhat hard) questions!
>
> > * when you drop compact storage and have some SSTables stored in
> compact storage and some SSTables stored normally, how do we present data
> correctly?
>
> There is no difference in SSTable format between normal and compact
> storage, the difference is only in how Table Metadata is represented. In
> other words, in code.
>
> * does upgradesstables and/or compaction migrate off the compact
> storage format into the normal format?
>
> I think above answers this: the only way to migrate off compact storage is
> to change table metadata, and accept the differences mentioned above.
>
> * how do we know it is safe to remove?
>
> Maybe if we're explicit about it and say that, for example, by version X
> compact storage will not be supported at all, and document all the
> pitfalls, people will migrate their applications, one at a time. There's
> been a few suggestions about how to mitigate, including client options in
> 10857, and introducing "hiding" columns in 15811, and many more that were
> discussed but not implemented.
>
> Hope this won't discourage further discussions, but I suggest we keep this
> thread focused on pros/cons of a suggested intermediate solution (bring
> back a minimal subset of CS), and collect ideas about migration, cutoff
> date and final deprecation in a separate thread or a document. I personally
> do not have a full answer to "when it is safe to remove", and it seems like
> the best we can do is to create a clear procedure.
>
> On Thu, Oct 22, 2020 at 6:38 PM Ekaterina Dimitrova <e.dimitrova@gmail.com
> >
> wrote:
>
> > Definitely that is not off the table.
> > Also, we talked this morning for a plan for additional testing to be
> > created as part of CASSANDRA-15588
> >
> > On Thu, 22 Oct 2020 at 12:33, David Capwell <dc...@gmail.com> wrote:
> >
> > > I am in favor of bringing it back, but I do feel we should also plan
> how
> > to
> > > get it removed as well.
> > >
> > > Few examples, would love to see fleshed out
> > >
> > > * when you drop compact storage and have some SSTables stored in
> compact
> > > storage and some SSTables stored normally, how do we present data
> > > correctly?
> > > * does upgradesstables and/or compaction migrate off the compact
> storage
> > > format into the normal format?
> > > * how do we know it is safe to remove?
> > >
> > >
> > >
> > > On Thu, Oct 22, 2020 at 6:55 AM Ekaterina Dimitrova <
> > e.dimitrova@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi Alex,
> > > > Thanks for bringing this up.
> > > > I am with you for returning part of the code back and considering
> this
> > > as a
> > > > bug.
> > > > I truly believe it is too late in the release to document changed
> > > behavior.
> > > > I think this contradicts with the project’s promise for no breaking
> > > > changes. This should have been documented in alpha.
> > > > Best regards,
> > > > Ekaterina
> > > >
> > > > On Thu, 22 Oct 2020 at 3:20, Oleksandr Petrov <
> > > oleksandr.petrov@gmail.com>
> > > > wrote:
> > > >
> > > > > Since this is an important subject, I thought it also makes sense
> to
> > > > start
> > > > > a mailing list thread.
> > > > >
> > > > > You may know that in 4.0 there was a plan to drop compact storage
> and
> > > > > related code. However, there are several behavioural changes
> related
> > to
> > > > > compact storage, and difference in visible behaviour between
> "normal"
> > > and
> > > > > compact tables are larger than most of us have anticipated: we
> first
> > > > > thought there’ll be only “appearing column” in dense case, but
> > there’s
> > > > > implicit nulls in clusterings thing, and row vs column deletion
> now,
> > > TTL,
> > > > > and more.
> > > > >
> > > > > Some of the recent issues on the subject are: CASSANDRA-16048
> > > > > <https://issues.apache.org/jira/browse/CASSANDRA-16048>, which
> > allows
> > > to
> > > > > ignore these differences. The other one was an attempt to improve
> the
> > > > user
> > > > > experience of anyone still using compact storage: CASSANDRA-15811
> > > > > <https://issues.apache.org/jira/browse/CASSANDRA-15811>.
> > > > >
> > > > > Easily reproducible differences are:
> > > > >
> > > > > (1) hidden columns show up, which breaks SELECT * queries
> > > > > (2) DELETE v and UPDATE v WITH TTL would result into row removals
> in
> > > > > non-dense compact tables (CASSANDRA-16069
> > > > > <https://issues.apache.org/jira/browse/CASSANDRA-16069>)
> > > > > (3) INSERT allows skipping clusterings, which are filled with nulls
> > by
> > > > > default.
> > > > >
> > > > > Some of these are tricky to support, as 15811 has shown. Anyone who
> > > might
> > > > > want to upgrade to 4.0 while still using compact storage might be
> > > > affected
> > > > > by being forced into one of these behaviours.
> > > > >
> > > > > Possible solutions are to document these behaviours, or to bring
> > back a
> > > > > minimal set of COMPACT STORAGE and keep supporting these in 4.0
> > > > >
> > > > > It looks like it is possible to leave some of the functionality
> > related
> > > > to
> > > > > DENSE flag and allow it to be present in 4.0, but only for these
> > three
> > > > (and
> > > > > potential related, however not directly visible) cases.
> > > > >
> > > > > You can find more details on the subject here:
> > > > > https://issues.apache.org/jira/browse/CASSANDRA-16217
> > > > >
> > > > > Thank you,
> > > > >
> > > > > -- Alex
> > > > >
> > > >
> > >
> >
>
>
> --
> alex p
>
Re: Minimal 4.0 COMPACT STORAGE backport
Posted by Oleksandr Petrov <ol...@gmail.com>.
These are some great (albeit somewhat hard) questions!
> * when you drop compact storage and have some SSTables stored in
compact storage and some SSTables stored normally, how do we present data
correctly?
There is no difference in SSTable format between normal and compact
storage, the difference is only in how Table Metadata is represented. In
other words, in code.
* does upgradesstables and/or compaction migrate off the compact
storage format into the normal format?
I think above answers this: the only way to migrate off compact storage is
to change table metadata, and accept the differences mentioned above.
* how do we know it is safe to remove?
Maybe if we're explicit about it and say that, for example, by version X
compact storage will not be supported at all, and document all the
pitfalls, people will migrate their applications, one at a time. There's
been a few suggestions about how to mitigate, including client options in
10857, and introducing "hiding" columns in 15811, and many more that were
discussed but not implemented.
Hope this won't discourage further discussions, but I suggest we keep this
thread focused on pros/cons of a suggested intermediate solution (bring
back a minimal subset of CS), and collect ideas about migration, cutoff
date and final deprecation in a separate thread or a document. I personally
do not have a full answer to "when it is safe to remove", and it seems like
the best we can do is to create a clear procedure.
On Thu, Oct 22, 2020 at 6:38 PM Ekaterina Dimitrova <e....@gmail.com>
wrote:
> Definitely that is not off the table.
> Also, we talked this morning for a plan for additional testing to be
> created as part of CASSANDRA-15588
>
> On Thu, 22 Oct 2020 at 12:33, David Capwell <dc...@gmail.com> wrote:
>
> > I am in favor of bringing it back, but I do feel we should also plan how
> to
> > get it removed as well.
> >
> > Few examples, would love to see fleshed out
> >
> > * when you drop compact storage and have some SSTables stored in compact
> > storage and some SSTables stored normally, how do we present data
> > correctly?
> > * does upgradesstables and/or compaction migrate off the compact storage
> > format into the normal format?
> > * how do we know it is safe to remove?
> >
> >
> >
> > On Thu, Oct 22, 2020 at 6:55 AM Ekaterina Dimitrova <
> e.dimitrova@gmail.com
> > >
> > wrote:
> >
> > > Hi Alex,
> > > Thanks for bringing this up.
> > > I am with you for returning part of the code back and considering this
> > as a
> > > bug.
> > > I truly believe it is too late in the release to document changed
> > behavior.
> > > I think this contradicts with the project’s promise for no breaking
> > > changes. This should have been documented in alpha.
> > > Best regards,
> > > Ekaterina
> > >
> > > On Thu, 22 Oct 2020 at 3:20, Oleksandr Petrov <
> > oleksandr.petrov@gmail.com>
> > > wrote:
> > >
> > > > Since this is an important subject, I thought it also makes sense to
> > > start
> > > > a mailing list thread.
> > > >
> > > > You may know that in 4.0 there was a plan to drop compact storage and
> > > > related code. However, there are several behavioural changes related
> to
> > > > compact storage, and difference in visible behaviour between "normal"
> > and
> > > > compact tables are larger than most of us have anticipated: we first
> > > > thought there’ll be only “appearing column” in dense case, but
> there’s
> > > > implicit nulls in clusterings thing, and row vs column deletion now,
> > TTL,
> > > > and more.
> > > >
> > > > Some of the recent issues on the subject are: CASSANDRA-16048
> > > > <https://issues.apache.org/jira/browse/CASSANDRA-16048>, which
> allows
> > to
> > > > ignore these differences. The other one was an attempt to improve the
> > > user
> > > > experience of anyone still using compact storage: CASSANDRA-15811
> > > > <https://issues.apache.org/jira/browse/CASSANDRA-15811>.
> > > >
> > > > Easily reproducible differences are:
> > > >
> > > > (1) hidden columns show up, which breaks SELECT * queries
> > > > (2) DELETE v and UPDATE v WITH TTL would result into row removals in
> > > > non-dense compact tables (CASSANDRA-16069
> > > > <https://issues.apache.org/jira/browse/CASSANDRA-16069>)
> > > > (3) INSERT allows skipping clusterings, which are filled with nulls
> by
> > > > default.
> > > >
> > > > Some of these are tricky to support, as 15811 has shown. Anyone who
> > might
> > > > want to upgrade to 4.0 while still using compact storage might be
> > > affected
> > > > by being forced into one of these behaviours.
> > > >
> > > > Possible solutions are to document these behaviours, or to bring
> back a
> > > > minimal set of COMPACT STORAGE and keep supporting these in 4.0
> > > >
> > > > It looks like it is possible to leave some of the functionality
> related
> > > to
> > > > DENSE flag and allow it to be present in 4.0, but only for these
> three
> > > (and
> > > > potential related, however not directly visible) cases.
> > > >
> > > > You can find more details on the subject here:
> > > > https://issues.apache.org/jira/browse/CASSANDRA-16217
> > > >
> > > > Thank you,
> > > >
> > > > -- Alex
> > > >
> > >
> >
>
--
alex p
Re: Minimal 4.0 COMPACT STORAGE backport
Posted by Ekaterina Dimitrova <e....@gmail.com>.
Definitely that is not off the table.
Also, we talked this morning for a plan for additional testing to be
created as part of CASSANDRA-15588
On Thu, 22 Oct 2020 at 12:33, David Capwell <dc...@gmail.com> wrote:
> I am in favor of bringing it back, but I do feel we should also plan how to
> get it removed as well.
>
> Few examples, would love to see fleshed out
>
> * when you drop compact storage and have some SSTables stored in compact
> storage and some SSTables stored normally, how do we present data
> correctly?
> * does upgradesstables and/or compaction migrate off the compact storage
> format into the normal format?
> * how do we know it is safe to remove?
>
>
>
> On Thu, Oct 22, 2020 at 6:55 AM Ekaterina Dimitrova <e.dimitrova@gmail.com
> >
> wrote:
>
> > Hi Alex,
> > Thanks for bringing this up.
> > I am with you for returning part of the code back and considering this
> as a
> > bug.
> > I truly believe it is too late in the release to document changed
> behavior.
> > I think this contradicts with the project’s promise for no breaking
> > changes. This should have been documented in alpha.
> > Best regards,
> > Ekaterina
> >
> > On Thu, 22 Oct 2020 at 3:20, Oleksandr Petrov <
> oleksandr.petrov@gmail.com>
> > wrote:
> >
> > > Since this is an important subject, I thought it also makes sense to
> > start
> > > a mailing list thread.
> > >
> > > You may know that in 4.0 there was a plan to drop compact storage and
> > > related code. However, there are several behavioural changes related to
> > > compact storage, and difference in visible behaviour between "normal"
> and
> > > compact tables are larger than most of us have anticipated: we first
> > > thought there’ll be only “appearing column” in dense case, but there’s
> > > implicit nulls in clusterings thing, and row vs column deletion now,
> TTL,
> > > and more.
> > >
> > > Some of the recent issues on the subject are: CASSANDRA-16048
> > > <https://issues.apache.org/jira/browse/CASSANDRA-16048>, which allows
> to
> > > ignore these differences. The other one was an attempt to improve the
> > user
> > > experience of anyone still using compact storage: CASSANDRA-15811
> > > <https://issues.apache.org/jira/browse/CASSANDRA-15811>.
> > >
> > > Easily reproducible differences are:
> > >
> > > (1) hidden columns show up, which breaks SELECT * queries
> > > (2) DELETE v and UPDATE v WITH TTL would result into row removals in
> > > non-dense compact tables (CASSANDRA-16069
> > > <https://issues.apache.org/jira/browse/CASSANDRA-16069>)
> > > (3) INSERT allows skipping clusterings, which are filled with nulls by
> > > default.
> > >
> > > Some of these are tricky to support, as 15811 has shown. Anyone who
> might
> > > want to upgrade to 4.0 while still using compact storage might be
> > affected
> > > by being forced into one of these behaviours.
> > >
> > > Possible solutions are to document these behaviours, or to bring back a
> > > minimal set of COMPACT STORAGE and keep supporting these in 4.0
> > >
> > > It looks like it is possible to leave some of the functionality related
> > to
> > > DENSE flag and allow it to be present in 4.0, but only for these three
> > (and
> > > potential related, however not directly visible) cases.
> > >
> > > You can find more details on the subject here:
> > > https://issues.apache.org/jira/browse/CASSANDRA-16217
> > >
> > > Thank you,
> > >
> > > -- Alex
> > >
> >
>
Re: Minimal 4.0 COMPACT STORAGE backport
Posted by David Capwell <dc...@gmail.com>.
I am in favor of bringing it back, but I do feel we should also plan how to
get it removed as well.
Few examples, would love to see fleshed out
* when you drop compact storage and have some SSTables stored in compact
storage and some SSTables stored normally, how do we present data correctly?
* does upgradesstables and/or compaction migrate off the compact storage
format into the normal format?
* how do we know it is safe to remove?
On Thu, Oct 22, 2020 at 6:55 AM Ekaterina Dimitrova <e....@gmail.com>
wrote:
> Hi Alex,
> Thanks for bringing this up.
> I am with you for returning part of the code back and considering this as a
> bug.
> I truly believe it is too late in the release to document changed behavior.
> I think this contradicts with the project’s promise for no breaking
> changes. This should have been documented in alpha.
> Best regards,
> Ekaterina
>
> On Thu, 22 Oct 2020 at 3:20, Oleksandr Petrov <ol...@gmail.com>
> wrote:
>
> > Since this is an important subject, I thought it also makes sense to
> start
> > a mailing list thread.
> >
> > You may know that in 4.0 there was a plan to drop compact storage and
> > related code. However, there are several behavioural changes related to
> > compact storage, and difference in visible behaviour between "normal" and
> > compact tables are larger than most of us have anticipated: we first
> > thought there’ll be only “appearing column” in dense case, but there’s
> > implicit nulls in clusterings thing, and row vs column deletion now, TTL,
> > and more.
> >
> > Some of the recent issues on the subject are: CASSANDRA-16048
> > <https://issues.apache.org/jira/browse/CASSANDRA-16048>, which allows to
> > ignore these differences. The other one was an attempt to improve the
> user
> > experience of anyone still using compact storage: CASSANDRA-15811
> > <https://issues.apache.org/jira/browse/CASSANDRA-15811>.
> >
> > Easily reproducible differences are:
> >
> > (1) hidden columns show up, which breaks SELECT * queries
> > (2) DELETE v and UPDATE v WITH TTL would result into row removals in
> > non-dense compact tables (CASSANDRA-16069
> > <https://issues.apache.org/jira/browse/CASSANDRA-16069>)
> > (3) INSERT allows skipping clusterings, which are filled with nulls by
> > default.
> >
> > Some of these are tricky to support, as 15811 has shown. Anyone who might
> > want to upgrade to 4.0 while still using compact storage might be
> affected
> > by being forced into one of these behaviours.
> >
> > Possible solutions are to document these behaviours, or to bring back a
> > minimal set of COMPACT STORAGE and keep supporting these in 4.0
> >
> > It looks like it is possible to leave some of the functionality related
> to
> > DENSE flag and allow it to be present in 4.0, but only for these three
> (and
> > potential related, however not directly visible) cases.
> >
> > You can find more details on the subject here:
> > https://issues.apache.org/jira/browse/CASSANDRA-16217
> >
> > Thank you,
> >
> > -- Alex
> >
>
Re: Minimal 4.0 COMPACT STORAGE backport
Posted by Ekaterina Dimitrova <e....@gmail.com>.
Hi Alex,
Thanks for bringing this up.
I am with you for returning part of the code back and considering this as a
bug.
I truly believe it is too late in the release to document changed behavior.
I think this contradicts with the project’s promise for no breaking
changes. This should have been documented in alpha.
Best regards,
Ekaterina
On Thu, 22 Oct 2020 at 3:20, Oleksandr Petrov <ol...@gmail.com>
wrote:
> Since this is an important subject, I thought it also makes sense to start
> a mailing list thread.
>
> You may know that in 4.0 there was a plan to drop compact storage and
> related code. However, there are several behavioural changes related to
> compact storage, and difference in visible behaviour between "normal" and
> compact tables are larger than most of us have anticipated: we first
> thought there’ll be only “appearing column” in dense case, but there’s
> implicit nulls in clusterings thing, and row vs column deletion now, TTL,
> and more.
>
> Some of the recent issues on the subject are: CASSANDRA-16048
> <https://issues.apache.org/jira/browse/CASSANDRA-16048>, which allows to
> ignore these differences. The other one was an attempt to improve the user
> experience of anyone still using compact storage: CASSANDRA-15811
> <https://issues.apache.org/jira/browse/CASSANDRA-15811>.
>
> Easily reproducible differences are:
>
> (1) hidden columns show up, which breaks SELECT * queries
> (2) DELETE v and UPDATE v WITH TTL would result into row removals in
> non-dense compact tables (CASSANDRA-16069
> <https://issues.apache.org/jira/browse/CASSANDRA-16069>)
> (3) INSERT allows skipping clusterings, which are filled with nulls by
> default.
>
> Some of these are tricky to support, as 15811 has shown. Anyone who might
> want to upgrade to 4.0 while still using compact storage might be affected
> by being forced into one of these behaviours.
>
> Possible solutions are to document these behaviours, or to bring back a
> minimal set of COMPACT STORAGE and keep supporting these in 4.0
>
> It looks like it is possible to leave some of the functionality related to
> DENSE flag and allow it to be present in 4.0, but only for these three (and
> potential related, however not directly visible) cases.
>
> You can find more details on the subject here:
> https://issues.apache.org/jira/browse/CASSANDRA-16217
>
> Thank you,
>
> -- Alex
>