You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Paulo Motta <pa...@apache.org> on 2021/08/20 15:43:08 UTC
Google Summer of Code Wrap Up: Add TTL support to nodetool snapshots
Hi everyone,
Just a heads up to the community that we're wrapping up the Google Summer
of Code project this year.
Abi Palagashvili worked with us in the last couple of months to provide TTL
support to nodetool snapshots on CASSANDRA-16789 <
https://issues.apache.org/jira/browse/CASSANDRA-16789>, under mine and
Stefan Miklosovic's mentorship. We're in the final round of review before
merging the feature and welcome anyone who wants to take a look in the
final patch and give feedback.
After this change is shipped in the next major release, clients can supply
an optional --ttl parameter during nodetool snapshot creation and Cassandra
will automatically clean up expired snapshots, avoiding the need for
external management of snapshot cleanup.
During the process of adding this feature we identified several improvement
areas and started an effort to modernize the snapshot module by
centralizing snapshot lifecycle management on a SnapshotManager class,
which is responsible for keeping track of active snapshots in memory and
periodically cleaning them up when they expire. Right now we're only
managing "expiring" snapshots in this class, but we plan to migrate the
legacy snapshot lifecycle management to this class in follow-up tickets to
decouple it from the keyspace and table management classes. We
significantly increased the test coverage of the snapshot lifecycle and
added in-jvm tests to verify the feature.
We plan to extend this feature before it's released on 4.1 by providing
support to pause/resume snapshot cleanup and also allow clients to supply
TTL to auto snapshots (those optionally created during truncation, table
drop or compaction), as well as integrate it with the ability of clearing
snapshots created since a specific date <
https://issues.apache.org/jira/browse/CASSANDRA-16860>. The parent task to
track future improvements in this area is CASSANDRA-16451 <
https://issues.apache.org/jira/browse/CASSANDRA-16451>.
We thank Abi very much for his effort during the project and hope he stays
around in the community!
Kind Regards,
Paulo
Re: Google Summer of Code Wrap Up: Add TTL support to nodetool snapshots
Posted by Benjamin Lerer <bl...@apache.org>.
Great work Abi, Paulo and Stefan!
Le sam. 21 août 2021 à 20:10, Ekaterina Dimitrova <e....@gmail.com> a
écrit :
> Thank you for all your time and efforts Abi, Paulo and Stefan!
>
> Abi, I hope you had also some fun around the work done and this was only
> the beginning of a continuous collaboration with the community.
>
> On Fri, 20 Aug 2021 at 11:50, Jonathan Ellis <jb...@gmail.com> wrote:
>
> > Thank you, Abi! And thanks to Stefan and Paulo for mentoring!
> >
> > On Fri, Aug 20, 2021 at 10:43 AM Paulo Motta <pa...@apache.org> wrote:
> >
> > > Hi everyone,
> > >
> > > Just a heads up to the community that we're wrapping up the Google
> Summer
> > > of Code project this year.
> > >
> > > Abi Palagashvili worked with us in the last couple of months to provide
> > TTL
> > > support to nodetool snapshots on CASSANDRA-16789 <
> > > https://issues.apache.org/jira/browse/CASSANDRA-16789>, under mine and
> > > Stefan Miklosovic's mentorship. We're in the final round of review
> before
> > > merging the feature and welcome anyone who wants to take a look in the
> > > final patch and give feedback.
> > >
> > > After this change is shipped in the next major release, clients can
> > supply
> > > an optional --ttl parameter during nodetool snapshot creation and
> > Cassandra
> > > will automatically clean up expired snapshots, avoiding the need for
> > > external management of snapshot cleanup.
> > >
> > > During the process of adding this feature we identified several
> > improvement
> > > areas and started an effort to modernize the snapshot module by
> > > centralizing snapshot lifecycle management on a SnapshotManager class,
> > > which is responsible for keeping track of active snapshots in memory
> and
> > > periodically cleaning them up when they expire. Right now we're only
> > > managing "expiring" snapshots in this class, but we plan to migrate the
> > > legacy snapshot lifecycle management to this class in follow-up tickets
> > to
> > > decouple it from the keyspace and table management classes. We
> > > significantly increased the test coverage of the snapshot lifecycle and
> > > added in-jvm tests to verify the feature.
> > >
> > > We plan to extend this feature before it's released on 4.1 by providing
> > > support to pause/resume snapshot cleanup and also allow clients to
> supply
> > > TTL to auto snapshots (those optionally created during truncation,
> table
> > > drop or compaction), as well as integrate it with the ability of
> clearing
> > > snapshots created since a specific date <
> > > https://issues.apache.org/jira/browse/CASSANDRA-16860>. The parent
> task
> > to
> > > track future improvements in this area is CASSANDRA-16451 <
> > > https://issues.apache.org/jira/browse/CASSANDRA-16451>.
> > >
> > > We thank Abi very much for his effort during the project and hope he
> > stays
> > > around in the community!
> > >
> > > Kind Regards,
> > >
> > > Paulo
> > >
> >
> >
> > --
> > Jonathan Ellis
> > co-founder, http://www.datastax.com
> > @spyced
> >
>
Re: Google Summer of Code Wrap Up: Add TTL support to nodetool snapshots
Posted by Ekaterina Dimitrova <e....@gmail.com>.
Thank you for all your time and efforts Abi, Paulo and Stefan!
Abi, I hope you had also some fun around the work done and this was only
the beginning of a continuous collaboration with the community.
On Fri, 20 Aug 2021 at 11:50, Jonathan Ellis <jb...@gmail.com> wrote:
> Thank you, Abi! And thanks to Stefan and Paulo for mentoring!
>
> On Fri, Aug 20, 2021 at 10:43 AM Paulo Motta <pa...@apache.org> wrote:
>
> > Hi everyone,
> >
> > Just a heads up to the community that we're wrapping up the Google Summer
> > of Code project this year.
> >
> > Abi Palagashvili worked with us in the last couple of months to provide
> TTL
> > support to nodetool snapshots on CASSANDRA-16789 <
> > https://issues.apache.org/jira/browse/CASSANDRA-16789>, under mine and
> > Stefan Miklosovic's mentorship. We're in the final round of review before
> > merging the feature and welcome anyone who wants to take a look in the
> > final patch and give feedback.
> >
> > After this change is shipped in the next major release, clients can
> supply
> > an optional --ttl parameter during nodetool snapshot creation and
> Cassandra
> > will automatically clean up expired snapshots, avoiding the need for
> > external management of snapshot cleanup.
> >
> > During the process of adding this feature we identified several
> improvement
> > areas and started an effort to modernize the snapshot module by
> > centralizing snapshot lifecycle management on a SnapshotManager class,
> > which is responsible for keeping track of active snapshots in memory and
> > periodically cleaning them up when they expire. Right now we're only
> > managing "expiring" snapshots in this class, but we plan to migrate the
> > legacy snapshot lifecycle management to this class in follow-up tickets
> to
> > decouple it from the keyspace and table management classes. We
> > significantly increased the test coverage of the snapshot lifecycle and
> > added in-jvm tests to verify the feature.
> >
> > We plan to extend this feature before it's released on 4.1 by providing
> > support to pause/resume snapshot cleanup and also allow clients to supply
> > TTL to auto snapshots (those optionally created during truncation, table
> > drop or compaction), as well as integrate it with the ability of clearing
> > snapshots created since a specific date <
> > https://issues.apache.org/jira/browse/CASSANDRA-16860>. The parent task
> to
> > track future improvements in this area is CASSANDRA-16451 <
> > https://issues.apache.org/jira/browse/CASSANDRA-16451>.
> >
> > We thank Abi very much for his effort during the project and hope he
> stays
> > around in the community!
> >
> > Kind Regards,
> >
> > Paulo
> >
>
>
> --
> Jonathan Ellis
> co-founder, http://www.datastax.com
> @spyced
>
Re: Google Summer of Code Wrap Up: Add TTL support to nodetool snapshots
Posted by Jonathan Ellis <jb...@gmail.com>.
Thank you, Abi! And thanks to Stefan and Paulo for mentoring!
On Fri, Aug 20, 2021 at 10:43 AM Paulo Motta <pa...@apache.org> wrote:
> Hi everyone,
>
> Just a heads up to the community that we're wrapping up the Google Summer
> of Code project this year.
>
> Abi Palagashvili worked with us in the last couple of months to provide TTL
> support to nodetool snapshots on CASSANDRA-16789 <
> https://issues.apache.org/jira/browse/CASSANDRA-16789>, under mine and
> Stefan Miklosovic's mentorship. We're in the final round of review before
> merging the feature and welcome anyone who wants to take a look in the
> final patch and give feedback.
>
> After this change is shipped in the next major release, clients can supply
> an optional --ttl parameter during nodetool snapshot creation and Cassandra
> will automatically clean up expired snapshots, avoiding the need for
> external management of snapshot cleanup.
>
> During the process of adding this feature we identified several improvement
> areas and started an effort to modernize the snapshot module by
> centralizing snapshot lifecycle management on a SnapshotManager class,
> which is responsible for keeping track of active snapshots in memory and
> periodically cleaning them up when they expire. Right now we're only
> managing "expiring" snapshots in this class, but we plan to migrate the
> legacy snapshot lifecycle management to this class in follow-up tickets to
> decouple it from the keyspace and table management classes. We
> significantly increased the test coverage of the snapshot lifecycle and
> added in-jvm tests to verify the feature.
>
> We plan to extend this feature before it's released on 4.1 by providing
> support to pause/resume snapshot cleanup and also allow clients to supply
> TTL to auto snapshots (those optionally created during truncation, table
> drop or compaction), as well as integrate it with the ability of clearing
> snapshots created since a specific date <
> https://issues.apache.org/jira/browse/CASSANDRA-16860>. The parent task to
> track future improvements in this area is CASSANDRA-16451 <
> https://issues.apache.org/jira/browse/CASSANDRA-16451>.
>
> We thank Abi very much for his effort during the project and hope he stays
> around in the community!
>
> Kind Regards,
>
> Paulo
>
--
Jonathan Ellis
co-founder, http://www.datastax.com
@spyced