You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Carl Mueller <ca...@smartthings.com.INVALID> on 2019/09/27 16:58:33 UTC
Re: upgrading from 2.x TWCS to 3.x TWCS
Is this still the official answer on TWCS 2.X --> 3.X upgrades? Pull the
code and recompile as a different package?
Can I just declare the necessary class and package namespace and delegate
to the actual main-codebase class?
On Mon, Nov 5, 2018 at 1:41 AM Oleksandr Shulgin <
oleksandr.shulgin@zalando.de> wrote:
> On Sat, Nov 3, 2018 at 1:13 AM Brian Spindler <br...@gmail.com>
> wrote:
>
>> That wasn't horrible at all. After testing, provided all goes well I can
>> submit this back to the main TWCS repo if you think it's worth it.
>>
>> Either way do you mind just reviewing briefly for obvious mistakes?
>>
>>
>> https://github.com/bspindler/twcs/commit/7ba388dbf41b1c9dc1b70661ad69273b258139da
>>
>
> About almost a year ago we were migrating from 2.1 to 3.0 and we figured
> out that Jeff's master branch didn't compile with 3.0, but the change to
> get it running was really minimal:
>
> https://github.com/a1exsh/twcs/commit/10ee91c6f409aa249c8d439f7670d8b997ab0869
>
> So we built that jar, added it to the packaged 3.0 and we were good to
> go. Maybe you might want to consider migrating in two steps: 2.1 -> 3.0,
> ALTER TABLE, upgradesstables, 3.0 -> 3.1.
>
> And huge thanks to Jeff for coming up with TWCS in the first place! :-)
>
> Cheers,
> --
> Alex
>
>
Re: upgrading from 2.x TWCS to 3.x TWCS
Posted by Carl Mueller <ca...@smartthings.com.INVALID>.
sweet
Ugh, for the test I just realized that sstableloader probably wont' produce
buckets corresponding with the actual insertion time of the data in twcs.
Well, we can still run the test.
On Mon, Sep 30, 2019 at 2:47 AM Oleksandr Shulgin <
oleksandr.shulgin@zalando.de> wrote:
> On Fri, Sep 27, 2019 at 7:39 PM Carl Mueller
> <ca...@smartthings.com.invalid> wrote:
>
>> So IF that delegate class would work:
>>
>> 1) create jar with the delegate class
>> 2) deploy jar along with upgrade on node
>> 3) once all nodes are upgraded, issue ALTER to change to the
>> org.apache.cassandra TWCS class.
>>
>
> Yes, this used to work for us in the same situation.
>
> will that trigger full recompaction?
>>
>
> Nope. :-)
>
> --
> Alex
>
>
Re: upgrading from 2.x TWCS to 3.x TWCS
Posted by Oleksandr Shulgin <ol...@zalando.de>.
On Fri, Sep 27, 2019 at 7:39 PM Carl Mueller
<ca...@smartthings.com.invalid> wrote:
> So IF that delegate class would work:
>
> 1) create jar with the delegate class
> 2) deploy jar along with upgrade on node
> 3) once all nodes are upgraded, issue ALTER to change to the
> org.apache.cassandra TWCS class.
>
Yes, this used to work for us in the same situation.
will that trigger full recompaction?
>
Nope. :-)
--
Alex
Re: upgrading from 2.x TWCS to 3.x TWCS
Posted by Jeff Jirsa <jj...@gmail.com>.
I suspect this will work, yes
(Sorry I can't make a proper upgrade path, sorta out of my hands)
On Fri, Sep 27, 2019 at 11:19 AM Carl Mueller
<ca...@smartthings.com.invalid> wrote:
> Oops, I think the getName() is an important thing:
>
> package com.jeffjirsa.cassandra.db.compaction;
>
> import org.apache.cassandra.db.ColumnFamilyStore;
>
> import java.util.Map;
>
> public class TimeWindowCompactionStrategy extends org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy {
> public TimeWindowCompactionStrategy(ColumnFamilyStore cfs, Map<String, String> options) {
> super(cfs,options);
> }
>
> public String getName()
> {
> return getClass().getSimpleName();
> }
> }
>
>
> On Fri, Sep 27, 2019 at 1:05 PM Carl Mueller <ca...@smartthings.com>
> wrote:
>
>> Or can we just do this safely in a side jar?
>>
>> package com.jeffjirsa.cassandra.db.compaction;
>>
>> import org.apache.cassandra.db.ColumnFamilyStore;
>> import org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy;
>>
>> import java.util.Map;
>>
>> public class TimeWindowCompactionStrategy extends TimeWindowCompactionStrategy {
>> public TimeWindowCompactionStrategy(ColumnFamilyStore cfs, Map<String, String> options) {
>> super(cfs,options);
>> }
>> }
>>
>>
>> On Fri, Sep 27, 2019 at 12:29 PM Carl Mueller <
>> carl.mueller@smartthings.com> wrote:
>>
>>> So IF that delegate class would work:
>>>
>>> 1) create jar with the delegate class
>>> 2) deploy jar along with upgrade on node
>>> 3) once all nodes are upgraded, issue ALTER to change to the
>>> org.apache.cassandra TWCS class.
>>>
>>> will that trigger full recompaction?
>>>
>>> On Fri, Sep 27, 2019 at 12:25 PM Carl Mueller <
>>> carl.mueller@smartthings.com> wrote:
>>>
>>>> For example (delegating all public methods from
>>>> AbstractCompactionStrategy and some from TimeWindowCompactionStrategy
>>>> EXCEPT getName() )::::
>>>>
>>>> package com.jeffjirsa.cassandra.db.compaction;
>>>>
>>>> public class TimeWindowCompactionStrategy extends
>>>> AbstractCompactionStrategy
>>>> {
>>>> org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy
>>>> delegate;
>>>>
>>>> public TimeWindowCompactionStrategy(ColumnFamilyStore cfs,
>>>> Map<String,String> options) {
>>>> delegate = new
>>>> org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy(cfs,options);
>>>> }
>>>>
>>>>
>>>> public Directories getDirectories() { return
>>>> delegate.getDirectories(); }
>>>> public synchronized void pause() { delegate.pause(); }
>>>> public synchronized void resume() { delegate.resume(); }
>>>> public synchronized void startup() { delegate.startup(); }
>>>> public synchronized void shutdown() { delegate.shutdown(); }
>>>>
>>>>
>>>>
>>>> public AbstractCompactionTask getNextBackgroundTask(int gcBefore) {
>>>> return delegate.getNextBackgroundTask(gcBefore); }
>>>> public synchronized Collection<AbstractCompactionTask>
>>>> getMaximalTask(int gcBefore, boolean splitOutput) { return
>>>> delegate.getMaximalTask(gcBefore, splitOutput); }
>>>> public synchronized AbstractCompactionTask
>>>> getUserDefinedTask(Collection<SSTableReader> sstables, int gcBefore) {
>>>> return getUserDefinedTask(sstables,gcBefore); }
>>>> public int getEstimatedRemainingTasks() { return
>>>> delegate.getEstimatedRemainingTasks(); }
>>>> public long getMaxSSTableBytes() { return
>>>> delegate.getMaxSSTableBytes(); }
>>>> public ScannerList getScanners(Collection<SSTableReader> sstables,
>>>> Range<Token> range) { return delegate.getScanners(sstables,range); }
>>>> public ScannerList getScanners(Collection<SSTableReader> sstables,
>>>> Collection<Range<Token>> ranges) { return delegate.getScanners(sstables,
>>>> ranges);}
>>>> public ScannerList getScanners(Collection<SSTableReader> toCompact)
>>>> { return delegate.getScanners(toCompact); }
>>>> protected boolean worthDroppingTombstones(SSTableReader sstable,
>>>> int gcBefore) { return delegate.worthDroppingTombstones(sstable,gcBefore); }
>>>>
>>>> public boolean shouldDefragment() { return
>>>> delegate.shouldDefragment(); }
>>>> public synchronized void replaceSSTables(Collection<SSTableReader>
>>>> removed, Collection<SSTableReader> added) {
>>>> delegate.replaceSSTables(removed,added); }
>>>> public Collection<Collection<SSTableReader>>
>>>> groupSSTablesForAntiCompaction(Collection<SSTableReader> sstablesToGroup) {
>>>> delegate.groupSSTablesForAntiCOmpaction(sstablesToGroup); }
>>>> public CompactionLogger.Strategy strategyLogger() { return
>>>> delegate.strategyLogger(); }
>>>> public SSTableMultiWriter createSSTableMultiWriter(Descriptor
>>>> descriptor,
>>>> long keyCount,
>>>> long repairedAt,
>>>> UUID
>>>> pendingRepair,
>>>> boolean
>>>> isTransient,
>>>>
>>>> MetadataCollector meta,
>>>>
>>>> SerializationHeader header,
>>>>
>>>> Collection<Index> indexes,
>>>>
>>>> LifecycleNewTracker lifecycleNewTracker) {
>>>> return delefate.createSSTableMultiWriter(descriptor, keyCount,
>>>> repairedAt, pendingRepair, isTransient, meta, header, indexes,
>>>> lifecycleNewTracker);
>>>> }
>>>>
>>>> public boolean supportsEarlyOpen() { return
>>>> delegate.supportsEarlyOpen(); }
>>>>
>>>>
>>>>
>>>>
>>>> public String getName() { return getClass().getSimpleName(); } //
>>>> don't delegate because this is probably the name used in the column family
>>>> definition.
>>>>
>>>>
>>>>
>>>>
>>>> public synchronized void addSSTable(SSTableReader sstable) {
>>>> delegate.addSSTable(sstable); }
>>>> public synchronized void addSSTables(Iterable<SSTableReader> added)
>>>> { delegate.addSSTables(added); }
>>>>
>>>> public synchronized void removeSSTable(SSTableReader sstable) {
>>>> delegate.removeSSTable(sstable); }
>>>> public void removeSSTables(Iterable<SSTableReader> removed) {
>>>> delegate.removeSSTables(removed); }
>>>>
>>>> public void metadataChanged(StatsMetadata oldMetadata,
>>>> SSTableReader sstable) { delegate.metadataChanges(oldMetadata,sstable); }
>>>>
>>>>
>>>> protected Set<SSTableReader> getSSTables() { return
>>>> delegate.getSSTables(); }
>>>> public String toString() {return delegate.toString(); }
>>>>
>>>>
>>>> public static Map<String, String> validateOptions(Map<String,
>>>> String> options) throws ConfigurationException { return
>>>> delegate.validateOptions(options); }
>>>>
>>>> }
>>>>
>>>> On Fri, Sep 27, 2019 at 11:58 AM Carl Mueller <
>>>> carl.mueller@smartthings.com> wrote:
>>>>
>>>>> Is this still the official answer on TWCS 2.X --> 3.X upgrades? Pull
>>>>> the code and recompile as a different package?
>>>>>
>>>>> Can I just declare the necessary class and package namespace and
>>>>> delegate to the actual main-codebase class?
>>>>>
>>>>>
>>>>> On Mon, Nov 5, 2018 at 1:41 AM Oleksandr Shulgin <
>>>>> oleksandr.shulgin@zalando.de> wrote:
>>>>>
>>>>>> On Sat, Nov 3, 2018 at 1:13 AM Brian Spindler <
>>>>>> brian.spindler@gmail.com> wrote:
>>>>>>
>>>>>>> That wasn't horrible at all. After testing, provided all goes well
>>>>>>> I can submit this back to the main TWCS repo if you think it's worth it.
>>>>>>>
>>>>>>> Either way do you mind just reviewing briefly for obvious mistakes?
>>>>>>>
>>>>>>>
>>>>>>> https://github.com/bspindler/twcs/commit/7ba388dbf41b1c9dc1b70661ad69273b258139da
>>>>>>>
>>>>>>
>>>>>> About almost a year ago we were migrating from 2.1 to 3.0 and we
>>>>>> figured out that Jeff's master branch didn't compile with 3.0, but the
>>>>>> change to get it running was really minimal:
>>>>>>
>>>>>> https://github.com/a1exsh/twcs/commit/10ee91c6f409aa249c8d439f7670d8b997ab0869
>>>>>>
>>>>>> So we built that jar, added it to the packaged 3.0 and we were good
>>>>>> to go. Maybe you might want to consider migrating in two steps: 2.1 ->
>>>>>> 3.0, ALTER TABLE, upgradesstables, 3.0 -> 3.1.
>>>>>>
>>>>>> And huge thanks to Jeff for coming up with TWCS in the first place!
>>>>>> :-)
>>>>>>
>>>>>> Cheers,
>>>>>> --
>>>>>> Alex
>>>>>>
>>>>>>
Re: upgrading from 2.x TWCS to 3.x TWCS
Posted by Carl Mueller <ca...@smartthings.com.INVALID>.
Oops, I think the getName() is an important thing:
package com.jeffjirsa.cassandra.db.compaction;
import org.apache.cassandra.db.ColumnFamilyStore;
import java.util.Map;
public class TimeWindowCompactionStrategy extends
org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy {
public TimeWindowCompactionStrategy(ColumnFamilyStore cfs,
Map<String, String> options) {
super(cfs,options);
}
public String getName()
{
return getClass().getSimpleName();
}
}
On Fri, Sep 27, 2019 at 1:05 PM Carl Mueller <ca...@smartthings.com>
wrote:
> Or can we just do this safely in a side jar?
>
> package com.jeffjirsa.cassandra.db.compaction;
>
> import org.apache.cassandra.db.ColumnFamilyStore;
> import org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy;
>
> import java.util.Map;
>
> public class TimeWindowCompactionStrategy extends TimeWindowCompactionStrategy {
> public TimeWindowCompactionStrategy(ColumnFamilyStore cfs, Map<String, String> options) {
> super(cfs,options);
> }
> }
>
>
> On Fri, Sep 27, 2019 at 12:29 PM Carl Mueller <
> carl.mueller@smartthings.com> wrote:
>
>> So IF that delegate class would work:
>>
>> 1) create jar with the delegate class
>> 2) deploy jar along with upgrade on node
>> 3) once all nodes are upgraded, issue ALTER to change to the
>> org.apache.cassandra TWCS class.
>>
>> will that trigger full recompaction?
>>
>> On Fri, Sep 27, 2019 at 12:25 PM Carl Mueller <
>> carl.mueller@smartthings.com> wrote:
>>
>>> For example (delegating all public methods from
>>> AbstractCompactionStrategy and some from TimeWindowCompactionStrategy
>>> EXCEPT getName() )::::
>>>
>>> package com.jeffjirsa.cassandra.db.compaction;
>>>
>>> public class TimeWindowCompactionStrategy extends
>>> AbstractCompactionStrategy
>>> {
>>> org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy
>>> delegate;
>>>
>>> public TimeWindowCompactionStrategy(ColumnFamilyStore cfs,
>>> Map<String,String> options) {
>>> delegate = new
>>> org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy(cfs,options);
>>> }
>>>
>>>
>>> public Directories getDirectories() { return
>>> delegate.getDirectories(); }
>>> public synchronized void pause() { delegate.pause(); }
>>> public synchronized void resume() { delegate.resume(); }
>>> public synchronized void startup() { delegate.startup(); }
>>> public synchronized void shutdown() { delegate.shutdown(); }
>>>
>>>
>>>
>>> public AbstractCompactionTask getNextBackgroundTask(int gcBefore) {
>>> return delegate.getNextBackgroundTask(gcBefore); }
>>> public synchronized Collection<AbstractCompactionTask>
>>> getMaximalTask(int gcBefore, boolean splitOutput) { return
>>> delegate.getMaximalTask(gcBefore, splitOutput); }
>>> public synchronized AbstractCompactionTask
>>> getUserDefinedTask(Collection<SSTableReader> sstables, int gcBefore) {
>>> return getUserDefinedTask(sstables,gcBefore); }
>>> public int getEstimatedRemainingTasks() { return
>>> delegate.getEstimatedRemainingTasks(); }
>>> public long getMaxSSTableBytes() { return
>>> delegate.getMaxSSTableBytes(); }
>>> public ScannerList getScanners(Collection<SSTableReader> sstables,
>>> Range<Token> range) { return delegate.getScanners(sstables,range); }
>>> public ScannerList getScanners(Collection<SSTableReader> sstables,
>>> Collection<Range<Token>> ranges) { return delegate.getScanners(sstables,
>>> ranges);}
>>> public ScannerList getScanners(Collection<SSTableReader> toCompact)
>>> { return delegate.getScanners(toCompact); }
>>> protected boolean worthDroppingTombstones(SSTableReader sstable, int
>>> gcBefore) { return delegate.worthDroppingTombstones(sstable,gcBefore); }
>>>
>>> public boolean shouldDefragment() { return
>>> delegate.shouldDefragment(); }
>>> public synchronized void replaceSSTables(Collection<SSTableReader>
>>> removed, Collection<SSTableReader> added) {
>>> delegate.replaceSSTables(removed,added); }
>>> public Collection<Collection<SSTableReader>>
>>> groupSSTablesForAntiCompaction(Collection<SSTableReader> sstablesToGroup) {
>>> delegate.groupSSTablesForAntiCOmpaction(sstablesToGroup); }
>>> public CompactionLogger.Strategy strategyLogger() { return
>>> delegate.strategyLogger(); }
>>> public SSTableMultiWriter createSSTableMultiWriter(Descriptor
>>> descriptor,
>>> long keyCount,
>>> long repairedAt,
>>> UUID
>>> pendingRepair,
>>> boolean
>>> isTransient,
>>> MetadataCollector
>>> meta,
>>>
>>> SerializationHeader header,
>>> Collection<Index>
>>> indexes,
>>>
>>> LifecycleNewTracker lifecycleNewTracker) {
>>> return delefate.createSSTableMultiWriter(descriptor, keyCount,
>>> repairedAt, pendingRepair, isTransient, meta, header, indexes,
>>> lifecycleNewTracker);
>>> }
>>>
>>> public boolean supportsEarlyOpen() { return
>>> delegate.supportsEarlyOpen(); }
>>>
>>>
>>>
>>>
>>> public String getName() { return getClass().getSimpleName(); } //
>>> don't delegate because this is probably the name used in the column family
>>> definition.
>>>
>>>
>>>
>>>
>>> public synchronized void addSSTable(SSTableReader sstable) {
>>> delegate.addSSTable(sstable); }
>>> public synchronized void addSSTables(Iterable<SSTableReader> added)
>>> { delegate.addSSTables(added); }
>>>
>>> public synchronized void removeSSTable(SSTableReader sstable) {
>>> delegate.removeSSTable(sstable); }
>>> public void removeSSTables(Iterable<SSTableReader> removed) {
>>> delegate.removeSSTables(removed); }
>>>
>>> public void metadataChanged(StatsMetadata oldMetadata, SSTableReader
>>> sstable) { delegate.metadataChanges(oldMetadata,sstable); }
>>>
>>>
>>> protected Set<SSTableReader> getSSTables() { return
>>> delegate.getSSTables(); }
>>> public String toString() {return delegate.toString(); }
>>>
>>>
>>> public static Map<String, String> validateOptions(Map<String,
>>> String> options) throws ConfigurationException { return
>>> delegate.validateOptions(options); }
>>>
>>> }
>>>
>>> On Fri, Sep 27, 2019 at 11:58 AM Carl Mueller <
>>> carl.mueller@smartthings.com> wrote:
>>>
>>>> Is this still the official answer on TWCS 2.X --> 3.X upgrades? Pull
>>>> the code and recompile as a different package?
>>>>
>>>> Can I just declare the necessary class and package namespace and
>>>> delegate to the actual main-codebase class?
>>>>
>>>>
>>>> On Mon, Nov 5, 2018 at 1:41 AM Oleksandr Shulgin <
>>>> oleksandr.shulgin@zalando.de> wrote:
>>>>
>>>>> On Sat, Nov 3, 2018 at 1:13 AM Brian Spindler <
>>>>> brian.spindler@gmail.com> wrote:
>>>>>
>>>>>> That wasn't horrible at all. After testing, provided all goes well I
>>>>>> can submit this back to the main TWCS repo if you think it's worth it.
>>>>>>
>>>>>> Either way do you mind just reviewing briefly for obvious mistakes?
>>>>>>
>>>>>>
>>>>>> https://github.com/bspindler/twcs/commit/7ba388dbf41b1c9dc1b70661ad69273b258139da
>>>>>>
>>>>>
>>>>> About almost a year ago we were migrating from 2.1 to 3.0 and we
>>>>> figured out that Jeff's master branch didn't compile with 3.0, but the
>>>>> change to get it running was really minimal:
>>>>>
>>>>> https://github.com/a1exsh/twcs/commit/10ee91c6f409aa249c8d439f7670d8b997ab0869
>>>>>
>>>>> So we built that jar, added it to the packaged 3.0 and we were good to
>>>>> go. Maybe you might want to consider migrating in two steps: 2.1 -> 3.0,
>>>>> ALTER TABLE, upgradesstables, 3.0 -> 3.1.
>>>>>
>>>>> And huge thanks to Jeff for coming up with TWCS in the first place! :-)
>>>>>
>>>>> Cheers,
>>>>> --
>>>>> Alex
>>>>>
>>>>>
Re: upgrading from 2.x TWCS to 3.x TWCS
Posted by Carl Mueller <ca...@smartthings.com.INVALID>.
Or can we just do this safely in a side jar?
package com.jeffjirsa.cassandra.db.compaction;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy;
import java.util.Map;
public class TimeWindowCompactionStrategy extends TimeWindowCompactionStrategy {
public TimeWindowCompactionStrategy(ColumnFamilyStore cfs,
Map<String, String> options) {
super(cfs,options);
}
}
On Fri, Sep 27, 2019 at 12:29 PM Carl Mueller <ca...@smartthings.com>
wrote:
> So IF that delegate class would work:
>
> 1) create jar with the delegate class
> 2) deploy jar along with upgrade on node
> 3) once all nodes are upgraded, issue ALTER to change to the
> org.apache.cassandra TWCS class.
>
> will that trigger full recompaction?
>
> On Fri, Sep 27, 2019 at 12:25 PM Carl Mueller <
> carl.mueller@smartthings.com> wrote:
>
>> For example (delegating all public methods from
>> AbstractCompactionStrategy and some from TimeWindowCompactionStrategy
>> EXCEPT getName() )::::
>>
>> package com.jeffjirsa.cassandra.db.compaction;
>>
>> public class TimeWindowCompactionStrategy extends
>> AbstractCompactionStrategy
>> {
>> org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy
>> delegate;
>>
>> public TimeWindowCompactionStrategy(ColumnFamilyStore cfs,
>> Map<String,String> options) {
>> delegate = new
>> org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy(cfs,options);
>> }
>>
>>
>> public Directories getDirectories() { return
>> delegate.getDirectories(); }
>> public synchronized void pause() { delegate.pause(); }
>> public synchronized void resume() { delegate.resume(); }
>> public synchronized void startup() { delegate.startup(); }
>> public synchronized void shutdown() { delegate.shutdown(); }
>>
>>
>>
>> public AbstractCompactionTask getNextBackgroundTask(int gcBefore) {
>> return delegate.getNextBackgroundTask(gcBefore); }
>> public synchronized Collection<AbstractCompactionTask>
>> getMaximalTask(int gcBefore, boolean splitOutput) { return
>> delegate.getMaximalTask(gcBefore, splitOutput); }
>> public synchronized AbstractCompactionTask
>> getUserDefinedTask(Collection<SSTableReader> sstables, int gcBefore) {
>> return getUserDefinedTask(sstables,gcBefore); }
>> public int getEstimatedRemainingTasks() { return
>> delegate.getEstimatedRemainingTasks(); }
>> public long getMaxSSTableBytes() { return
>> delegate.getMaxSSTableBytes(); }
>> public ScannerList getScanners(Collection<SSTableReader> sstables,
>> Range<Token> range) { return delegate.getScanners(sstables,range); }
>> public ScannerList getScanners(Collection<SSTableReader> sstables,
>> Collection<Range<Token>> ranges) { return delegate.getScanners(sstables,
>> ranges);}
>> public ScannerList getScanners(Collection<SSTableReader> toCompact) {
>> return delegate.getScanners(toCompact); }
>> protected boolean worthDroppingTombstones(SSTableReader sstable, int
>> gcBefore) { return delegate.worthDroppingTombstones(sstable,gcBefore); }
>>
>> public boolean shouldDefragment() { return
>> delegate.shouldDefragment(); }
>> public synchronized void replaceSSTables(Collection<SSTableReader>
>> removed, Collection<SSTableReader> added) {
>> delegate.replaceSSTables(removed,added); }
>> public Collection<Collection<SSTableReader>>
>> groupSSTablesForAntiCompaction(Collection<SSTableReader> sstablesToGroup) {
>> delegate.groupSSTablesForAntiCOmpaction(sstablesToGroup); }
>> public CompactionLogger.Strategy strategyLogger() { return
>> delegate.strategyLogger(); }
>> public SSTableMultiWriter createSSTableMultiWriter(Descriptor
>> descriptor,
>> long keyCount,
>> long repairedAt,
>> UUID pendingRepair,
>> boolean
>> isTransient,
>> MetadataCollector
>> meta,
>>
>> SerializationHeader header,
>> Collection<Index>
>> indexes,
>>
>> LifecycleNewTracker lifecycleNewTracker) {
>> return delefate.createSSTableMultiWriter(descriptor, keyCount,
>> repairedAt, pendingRepair, isTransient, meta, header, indexes,
>> lifecycleNewTracker);
>> }
>>
>> public boolean supportsEarlyOpen() { return
>> delegate.supportsEarlyOpen(); }
>>
>>
>>
>>
>> public String getName() { return getClass().getSimpleName(); } //
>> don't delegate because this is probably the name used in the column family
>> definition.
>>
>>
>>
>>
>> public synchronized void addSSTable(SSTableReader sstable) {
>> delegate.addSSTable(sstable); }
>> public synchronized void addSSTables(Iterable<SSTableReader> added) {
>> delegate.addSSTables(added); }
>>
>> public synchronized void removeSSTable(SSTableReader sstable) {
>> delegate.removeSSTable(sstable); }
>> public void removeSSTables(Iterable<SSTableReader> removed) {
>> delegate.removeSSTables(removed); }
>>
>> public void metadataChanged(StatsMetadata oldMetadata, SSTableReader
>> sstable) { delegate.metadataChanges(oldMetadata,sstable); }
>>
>>
>> protected Set<SSTableReader> getSSTables() { return
>> delegate.getSSTables(); }
>> public String toString() {return delegate.toString(); }
>>
>>
>> public static Map<String, String> validateOptions(Map<String, String>
>> options) throws ConfigurationException { return
>> delegate.validateOptions(options); }
>>
>> }
>>
>> On Fri, Sep 27, 2019 at 11:58 AM Carl Mueller <
>> carl.mueller@smartthings.com> wrote:
>>
>>> Is this still the official answer on TWCS 2.X --> 3.X upgrades? Pull the
>>> code and recompile as a different package?
>>>
>>> Can I just declare the necessary class and package namespace and
>>> delegate to the actual main-codebase class?
>>>
>>>
>>> On Mon, Nov 5, 2018 at 1:41 AM Oleksandr Shulgin <
>>> oleksandr.shulgin@zalando.de> wrote:
>>>
>>>> On Sat, Nov 3, 2018 at 1:13 AM Brian Spindler <br...@gmail.com>
>>>> wrote:
>>>>
>>>>> That wasn't horrible at all. After testing, provided all goes well I
>>>>> can submit this back to the main TWCS repo if you think it's worth it.
>>>>>
>>>>> Either way do you mind just reviewing briefly for obvious mistakes?
>>>>>
>>>>>
>>>>> https://github.com/bspindler/twcs/commit/7ba388dbf41b1c9dc1b70661ad69273b258139da
>>>>>
>>>>
>>>> About almost a year ago we were migrating from 2.1 to 3.0 and we
>>>> figured out that Jeff's master branch didn't compile with 3.0, but the
>>>> change to get it running was really minimal:
>>>>
>>>> https://github.com/a1exsh/twcs/commit/10ee91c6f409aa249c8d439f7670d8b997ab0869
>>>>
>>>> So we built that jar, added it to the packaged 3.0 and we were good to
>>>> go. Maybe you might want to consider migrating in two steps: 2.1 -> 3.0,
>>>> ALTER TABLE, upgradesstables, 3.0 -> 3.1.
>>>>
>>>> And huge thanks to Jeff for coming up with TWCS in the first place! :-)
>>>>
>>>> Cheers,
>>>> --
>>>> Alex
>>>>
>>>>
Re: upgrading from 2.x TWCS to 3.x TWCS
Posted by Carl Mueller <ca...@smartthings.com.INVALID>.
So IF that delegate class would work:
1) create jar with the delegate class
2) deploy jar along with upgrade on node
3) once all nodes are upgraded, issue ALTER to change to the
org.apache.cassandra TWCS class.
will that trigger full recompaction?
On Fri, Sep 27, 2019 at 12:25 PM Carl Mueller <ca...@smartthings.com>
wrote:
> For example (delegating all public methods from AbstractCompactionStrategy
> and some from TimeWindowCompactionStrategy EXCEPT getName() )::::
>
> package com.jeffjirsa.cassandra.db.compaction;
>
> public class TimeWindowCompactionStrategy extends
> AbstractCompactionStrategy
> {
> org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy
> delegate;
>
> public TimeWindowCompactionStrategy(ColumnFamilyStore cfs,
> Map<String,String> options) {
> delegate = new
> org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy(cfs,options);
> }
>
>
> public Directories getDirectories() { return
> delegate.getDirectories(); }
> public synchronized void pause() { delegate.pause(); }
> public synchronized void resume() { delegate.resume(); }
> public synchronized void startup() { delegate.startup(); }
> public synchronized void shutdown() { delegate.shutdown(); }
>
>
>
> public AbstractCompactionTask getNextBackgroundTask(int gcBefore) {
> return delegate.getNextBackgroundTask(gcBefore); }
> public synchronized Collection<AbstractCompactionTask>
> getMaximalTask(int gcBefore, boolean splitOutput) { return
> delegate.getMaximalTask(gcBefore, splitOutput); }
> public synchronized AbstractCompactionTask
> getUserDefinedTask(Collection<SSTableReader> sstables, int gcBefore) {
> return getUserDefinedTask(sstables,gcBefore); }
> public int getEstimatedRemainingTasks() { return
> delegate.getEstimatedRemainingTasks(); }
> public long getMaxSSTableBytes() { return
> delegate.getMaxSSTableBytes(); }
> public ScannerList getScanners(Collection<SSTableReader> sstables,
> Range<Token> range) { return delegate.getScanners(sstables,range); }
> public ScannerList getScanners(Collection<SSTableReader> sstables,
> Collection<Range<Token>> ranges) { return delegate.getScanners(sstables,
> ranges);}
> public ScannerList getScanners(Collection<SSTableReader> toCompact) {
> return delegate.getScanners(toCompact); }
> protected boolean worthDroppingTombstones(SSTableReader sstable, int
> gcBefore) { return delegate.worthDroppingTombstones(sstable,gcBefore); }
>
> public boolean shouldDefragment() { return
> delegate.shouldDefragment(); }
> public synchronized void replaceSSTables(Collection<SSTableReader>
> removed, Collection<SSTableReader> added) {
> delegate.replaceSSTables(removed,added); }
> public Collection<Collection<SSTableReader>>
> groupSSTablesForAntiCompaction(Collection<SSTableReader> sstablesToGroup) {
> delegate.groupSSTablesForAntiCOmpaction(sstablesToGroup); }
> public CompactionLogger.Strategy strategyLogger() { return
> delegate.strategyLogger(); }
> public SSTableMultiWriter createSSTableMultiWriter(Descriptor
> descriptor,
> long keyCount,
> long repairedAt,
> UUID pendingRepair,
> boolean isTransient,
> MetadataCollector
> meta,
> SerializationHeader
> header,
> Collection<Index>
> indexes,
> LifecycleNewTracker
> lifecycleNewTracker) {
> return delefate.createSSTableMultiWriter(descriptor, keyCount,
> repairedAt, pendingRepair, isTransient, meta, header, indexes,
> lifecycleNewTracker);
> }
>
> public boolean supportsEarlyOpen() { return
> delegate.supportsEarlyOpen(); }
>
>
>
>
> public String getName() { return getClass().getSimpleName(); } //
> don't delegate because this is probably the name used in the column family
> definition.
>
>
>
>
> public synchronized void addSSTable(SSTableReader sstable) {
> delegate.addSSTable(sstable); }
> public synchronized void addSSTables(Iterable<SSTableReader> added) {
> delegate.addSSTables(added); }
>
> public synchronized void removeSSTable(SSTableReader sstable) {
> delegate.removeSSTable(sstable); }
> public void removeSSTables(Iterable<SSTableReader> removed) {
> delegate.removeSSTables(removed); }
>
> public void metadataChanged(StatsMetadata oldMetadata, SSTableReader
> sstable) { delegate.metadataChanges(oldMetadata,sstable); }
>
>
> protected Set<SSTableReader> getSSTables() { return
> delegate.getSSTables(); }
> public String toString() {return delegate.toString(); }
>
>
> public static Map<String, String> validateOptions(Map<String, String>
> options) throws ConfigurationException { return
> delegate.validateOptions(options); }
>
> }
>
> On Fri, Sep 27, 2019 at 11:58 AM Carl Mueller <
> carl.mueller@smartthings.com> wrote:
>
>> Is this still the official answer on TWCS 2.X --> 3.X upgrades? Pull the
>> code and recompile as a different package?
>>
>> Can I just declare the necessary class and package namespace and delegate
>> to the actual main-codebase class?
>>
>>
>> On Mon, Nov 5, 2018 at 1:41 AM Oleksandr Shulgin <
>> oleksandr.shulgin@zalando.de> wrote:
>>
>>> On Sat, Nov 3, 2018 at 1:13 AM Brian Spindler <br...@gmail.com>
>>> wrote:
>>>
>>>> That wasn't horrible at all. After testing, provided all goes well I
>>>> can submit this back to the main TWCS repo if you think it's worth it.
>>>>
>>>> Either way do you mind just reviewing briefly for obvious mistakes?
>>>>
>>>>
>>>> https://github.com/bspindler/twcs/commit/7ba388dbf41b1c9dc1b70661ad69273b258139da
>>>>
>>>
>>> About almost a year ago we were migrating from 2.1 to 3.0 and we figured
>>> out that Jeff's master branch didn't compile with 3.0, but the change to
>>> get it running was really minimal:
>>>
>>> https://github.com/a1exsh/twcs/commit/10ee91c6f409aa249c8d439f7670d8b997ab0869
>>>
>>> So we built that jar, added it to the packaged 3.0 and we were good to
>>> go. Maybe you might want to consider migrating in two steps: 2.1 -> 3.0,
>>> ALTER TABLE, upgradesstables, 3.0 -> 3.1.
>>>
>>> And huge thanks to Jeff for coming up with TWCS in the first place! :-)
>>>
>>> Cheers,
>>> --
>>> Alex
>>>
>>>
Re: upgrading from 2.x TWCS to 3.x TWCS
Posted by Carl Mueller <ca...@smartthings.com.INVALID>.
For example (delegating all public methods from AbstractCompactionStrategy
and some from TimeWindowCompactionStrategy EXCEPT getName() )::::
package com.jeffjirsa.cassandra.db.compaction;
public class TimeWindowCompactionStrategy extends AbstractCompactionStrategy
{
org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy
delegate;
public TimeWindowCompactionStrategy(ColumnFamilyStore cfs,
Map<String,String> options) {
delegate = new
org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy(cfs,options);
}
public Directories getDirectories() { return delegate.getDirectories();
}
public synchronized void pause() { delegate.pause(); }
public synchronized void resume() { delegate.resume(); }
public synchronized void startup() { delegate.startup(); }
public synchronized void shutdown() { delegate.shutdown(); }
public AbstractCompactionTask getNextBackgroundTask(int gcBefore) {
return delegate.getNextBackgroundTask(gcBefore); }
public synchronized Collection<AbstractCompactionTask>
getMaximalTask(int gcBefore, boolean splitOutput) { return
delegate.getMaximalTask(gcBefore, splitOutput); }
public synchronized AbstractCompactionTask
getUserDefinedTask(Collection<SSTableReader> sstables, int gcBefore) {
return getUserDefinedTask(sstables,gcBefore); }
public int getEstimatedRemainingTasks() { return
delegate.getEstimatedRemainingTasks(); }
public long getMaxSSTableBytes() { return
delegate.getMaxSSTableBytes(); }
public ScannerList getScanners(Collection<SSTableReader> sstables,
Range<Token> range) { return delegate.getScanners(sstables,range); }
public ScannerList getScanners(Collection<SSTableReader> sstables,
Collection<Range<Token>> ranges) { return delegate.getScanners(sstables,
ranges);}
public ScannerList getScanners(Collection<SSTableReader> toCompact) {
return delegate.getScanners(toCompact); }
protected boolean worthDroppingTombstones(SSTableReader sstable, int
gcBefore) { return delegate.worthDroppingTombstones(sstable,gcBefore); }
public boolean shouldDefragment() { return delegate.shouldDefragment();
}
public synchronized void replaceSSTables(Collection<SSTableReader>
removed, Collection<SSTableReader> added) {
delegate.replaceSSTables(removed,added); }
public Collection<Collection<SSTableReader>>
groupSSTablesForAntiCompaction(Collection<SSTableReader> sstablesToGroup) {
delegate.groupSSTablesForAntiCOmpaction(sstablesToGroup); }
public CompactionLogger.Strategy strategyLogger() { return
delegate.strategyLogger(); }
public SSTableMultiWriter createSSTableMultiWriter(Descriptor
descriptor,
long keyCount,
long repairedAt,
UUID pendingRepair,
boolean isTransient,
MetadataCollector
meta,
SerializationHeader
header,
Collection<Index>
indexes,
LifecycleNewTracker
lifecycleNewTracker) {
return delefate.createSSTableMultiWriter(descriptor, keyCount,
repairedAt, pendingRepair, isTransient, meta, header, indexes,
lifecycleNewTracker);
}
public boolean supportsEarlyOpen() { return
delegate.supportsEarlyOpen(); }
public String getName() { return getClass().getSimpleName(); } // don't
delegate because this is probably the name used in the column family
definition.
public synchronized void addSSTable(SSTableReader sstable) {
delegate.addSSTable(sstable); }
public synchronized void addSSTables(Iterable<SSTableReader> added) {
delegate.addSSTables(added); }
public synchronized void removeSSTable(SSTableReader sstable) {
delegate.removeSSTable(sstable); }
public void removeSSTables(Iterable<SSTableReader> removed) {
delegate.removeSSTables(removed); }
public void metadataChanged(StatsMetadata oldMetadata, SSTableReader
sstable) { delegate.metadataChanges(oldMetadata,sstable); }
protected Set<SSTableReader> getSSTables() { return
delegate.getSSTables(); }
public String toString() {return delegate.toString(); }
public static Map<String, String> validateOptions(Map<String, String>
options) throws ConfigurationException { return
delegate.validateOptions(options); }
}
On Fri, Sep 27, 2019 at 11:58 AM Carl Mueller <ca...@smartthings.com>
wrote:
> Is this still the official answer on TWCS 2.X --> 3.X upgrades? Pull the
> code and recompile as a different package?
>
> Can I just declare the necessary class and package namespace and delegate
> to the actual main-codebase class?
>
>
> On Mon, Nov 5, 2018 at 1:41 AM Oleksandr Shulgin <
> oleksandr.shulgin@zalando.de> wrote:
>
>> On Sat, Nov 3, 2018 at 1:13 AM Brian Spindler <br...@gmail.com>
>> wrote:
>>
>>> That wasn't horrible at all. After testing, provided all goes well I
>>> can submit this back to the main TWCS repo if you think it's worth it.
>>>
>>> Either way do you mind just reviewing briefly for obvious mistakes?
>>>
>>>
>>> https://github.com/bspindler/twcs/commit/7ba388dbf41b1c9dc1b70661ad69273b258139da
>>>
>>
>> About almost a year ago we were migrating from 2.1 to 3.0 and we figured
>> out that Jeff's master branch didn't compile with 3.0, but the change to
>> get it running was really minimal:
>>
>> https://github.com/a1exsh/twcs/commit/10ee91c6f409aa249c8d439f7670d8b997ab0869
>>
>> So we built that jar, added it to the packaged 3.0 and we were good to
>> go. Maybe you might want to consider migrating in two steps: 2.1 -> 3.0,
>> ALTER TABLE, upgradesstables, 3.0 -> 3.1.
>>
>> And huge thanks to Jeff for coming up with TWCS in the first place! :-)
>>
>> Cheers,
>> --
>> Alex
>>
>>