You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by "Runtian (via GitHub)" <gi...@apache.org> on 2024/11/20 20:52:19 UTC
[PR] ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Runtian opened a new pull request, #3700:
URL: https://github.com/apache/cassandra/pull/3700
ReadCommandController should close fast to avoid deadlock when buildi…
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSANDRA-19564 ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "maedhroz (via GitHub)" <gi...@apache.org>.
maedhroz commented on PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#issuecomment-3433320452
Only test failure I'm seeing so far is just a result of not having [CASSANDRA-18440](https://issues.apache.org/jira/browse/CASSANDRA-18440)...this needs a rebase...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSANDRA-19564 ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "maedhroz (via GitHub)" <gi...@apache.org>.
maedhroz commented on code in PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#discussion_r2452684071
##########
test/distributed/org/apache/cassandra/distributed/test/SecondaryIndexTest.java:
##########
@@ -122,4 +132,53 @@ public void test_only_coordinator_chooses_index_for_query()
});
}
}
+
+ @Test
+ public void test_secondary_rebuild_with_small_memtable_memory()
+ {
+ // populate data
+ for (int i = 0 ; i < 100 ; ++i)
+ cluster.coordinator(1).execute(String.format("INSERT INTO %s (k, v) VALUES (?, ?)", tableName), ConsistencyLevel.ALL, i, generateRandomString(50000));
+
+ cluster.forEach(i -> i.flush(KEYSPACE));
+
+ // restart node 1 with small memtable allocation so that index rebuild will cause memtable flush which will need
+ // to reclaim the memory. see CASSANDRA-19564
+ waitOn(cluster.get(1).shutdown());
+ cluster.get(1).config().set("memtable_heap_space", "1MiB");
Review Comment:
See comment below...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "maedhroz (via GitHub)" <gi...@apache.org>.
maedhroz commented on code in PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#discussion_r2427445743
##########
src/java/org/apache/cassandra/index/SecondaryIndexManager.java:
##########
@@ -33,6 +33,8 @@
import com.google.common.collect.*;
import com.google.common.primitives.Longs;
import com.google.common.util.concurrent.FutureCallback;
+
+import org.apache.cassandra.db.partitions.ImmutableBTreePartition;
Review Comment:
nit: Move `cassandra` imports down to the proper position
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSANDRA-19564 ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "maedhroz (via GitHub)" <gi...@apache.org>.
maedhroz commented on code in PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#discussion_r2452637476
##########
test/distributed/org/apache/cassandra/distributed/test/SecondaryIndexTest.java:
##########
@@ -122,4 +133,44 @@ public void test_only_coordinator_chooses_index_for_query()
});
}
}
+
+ @Test
+ public void test_secondary_rebuild_with_small_memtable_memory()
+ {
+ // populate data
+ for (int i = 0 ; i < 100 ; ++i)
+ cluster.coordinator(1).execute(String.format("INSERT INTO %s (k, v) VALUES (?, ?)", tableName), ConsistencyLevel.ALL, i, ValueGenerator.randomString(new Random(), 50000));
+
+ cluster.forEach(i -> i.flush(KEYSPACE));
+
+ // restart node 1 with small memtable allocation so that index rebuild will cause memtable flush which will need
+ // to reclaim the memory. see CASSANDRA-19564
+ waitOn(cluster.get(1).shutdown());
+ Object originalMemTableHeapSpace = cluster.get(1).config().get("memtable_heap_space");
+ cluster.get(1).config().set("memtable_heap_space", "1MiB");
+ cluster.get(1).startup();
+ String tableNameWithoutKeyspaceName = tableName.split("\\.")[1];
+ String indexName = String.format("v_index_%d", seq.get());
+ Runnable task = cluster.get(1).runsOnInstance(
+ () -> {
+ ColumnFamilyStore cfs = Keyspace.open(KEYSPACE).getColumnFamilyStore(tableNameWithoutKeyspaceName);
+ cfs.indexManager.rebuildIndexesBlocking(Sets.newHashSet(Arrays.asList(indexName)));
+ }
+ );
+ ExecutorService es = Executors.newFixedThreadPool(1);
+ Future<?> future = es.submit(task);
+ try
+ {
+ future.get(30, TimeUnit.SECONDS);
+ }
+ catch (Exception e)
+ {
+ e.printStackTrace();
+ Assert.fail("Rebuild should finish within 30 seconds without issue.");
+ }
+ finally
+ {
+ cluster.get(1).config().set("memtable_heap_space", originalMemTableHeapSpace);
Review Comment:
There would still need to be a restart here though, right? If this test runs first, the `memtable_heap_space` change won't be picked up by subsequent tests, as initial node starts happen in `@BeforeClass`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "maedhroz (via GitHub)" <gi...@apache.org>.
maedhroz commented on code in PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#discussion_r2427520905
##########
test/distributed/org/apache/cassandra/distributed/test/SecondaryIndexTest.java:
##########
@@ -122,4 +132,53 @@ public void test_only_coordinator_chooses_index_for_query()
});
}
}
+
+ @Test
+ public void test_secondary_rebuild_with_small_memtable_memory()
+ {
+ // populate data
+ for (int i = 0 ; i < 100 ; ++i)
+ cluster.coordinator(1).execute(String.format("INSERT INTO %s (k, v) VALUES (?, ?)", tableName), ConsistencyLevel.ALL, i, generateRandomString(50000));
Review Comment:
We have random string generation in a few other places, like `ValueGenerator#randomString()`, `TombstonesWithIndexedSSTableTest#makeRandomString()`, etc. Might be nice to just pull at least this new thing into `ValueGenerator` (perhaps parameterize) and remove `generateRandomString()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "jfitz1 (via GitHub)" <gi...@apache.org>.
jfitz1 commented on PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#issuecomment-3109998367
+1 is this fix getting into the 4.1.x release or only as a patch? Running into this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSANDRA-19564 ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "jaydeepkumar1984 (via GitHub)" <gi...@apache.org>.
jaydeepkumar1984 closed pull request #3700: CASSANDRA-19564 ReadCommandController should close fast to avoid deadlock when buildi…
URL: https://github.com/apache/cassandra/pull/3700
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "maedhroz (via GitHub)" <gi...@apache.org>.
maedhroz commented on code in PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#discussion_r2427530138
##########
src/java/org/apache/cassandra/index/SecondaryIndexManager.java:
##########
@@ -918,14 +920,21 @@ public void indexPartition(DecoratedKey key, Set<Index> indexes, int pageSize, R
SinglePartitionPager pager = new SinglePartitionPager(cmd, null, ProtocolVersion.CURRENT);
while (!pager.isExhausted())
{
+ UnfilteredRowIterator partition;
try (ReadExecutionController controller = cmd.executionController();
- WriteContext ctx = keyspace.getWriteHandler().createContextForIndexing();
UnfilteredPartitionIterator page = pager.fetchPageUnfiltered(baseCfs.metadata(), pageSize, controller))
{
if (!page.hasNext())
break;
- try (UnfilteredRowIterator partition = page.next())
+ try (UnfilteredRowIterator onePartition = page.next())
+ {
+ partition = ImmutableBTreePartition.create(onePartition).unfilteredIterator();
+ }
+ }
+
+ try (WriteContext ctx = keyspace.getWriteHandler().createContextForIndexing())
Review Comment:
At the end of this `try` block, you might want to add:
```
finally
{
if (partition != null)
partition.close();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSANDRA-19564 ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "maedhroz (via GitHub)" <gi...@apache.org>.
maedhroz commented on code in PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#discussion_r2452626904
##########
test/distributed/org/apache/cassandra/distributed/test/SecondaryIndexTest.java:
##########
@@ -122,4 +133,44 @@ public void test_only_coordinator_chooses_index_for_query()
});
}
}
+
+ @Test
+ public void test_secondary_rebuild_with_small_memtable_memory()
+ {
+ // populate data
+ for (int i = 0 ; i < 100 ; ++i)
+ cluster.coordinator(1).execute(String.format("INSERT INTO %s (k, v) VALUES (?, ?)", tableName), ConsistencyLevel.ALL, i, ValueGenerator.randomString(new Random(), 50000));
Review Comment:
nit: I guess you could probably just use the same `Random` instance for al these writes...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSANDRA-19564 ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "maedhroz (via GitHub)" <gi...@apache.org>.
maedhroz commented on PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#issuecomment-3434562694
Need 5.0 and trunk patches and CI runs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "maedhroz (via GitHub)" <gi...@apache.org>.
maedhroz commented on code in PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#discussion_r2427521601
##########
test/distributed/org/apache/cassandra/distributed/test/SecondaryIndexTest.java:
##########
@@ -122,4 +132,53 @@ public void test_only_coordinator_chooses_index_for_query()
});
}
}
+
+ @Test
+ public void test_secondary_rebuild_with_small_memtable_memory()
+ {
+ // populate data
+ for (int i = 0 ; i < 100 ; ++i)
+ cluster.coordinator(1).execute(String.format("INSERT INTO %s (k, v) VALUES (?, ?)", tableName), ConsistencyLevel.ALL, i, generateRandomString(50000));
+
+ cluster.forEach(i -> i.flush(KEYSPACE));
+
+ // restart node 1 with small memtable allocation so that index rebuild will cause memtable flush which will need
+ // to reclaim the memory. see CASSANDRA-19564
+ waitOn(cluster.get(1).shutdown());
+ cluster.get(1).config().set("memtable_heap_space", "1MiB");
+ cluster.get(1).startup();
+ String tableNameWithoutKeyspaceName = tableName.split("\\.")[1];
+ String indexName = String.format("v_index_%d", seq.get());
+ Runnable task = cluster.get(1).runsOnInstance(
+ () -> {
+ ColumnFamilyStore cfs = Keyspace.open(KEYSPACE).getColumnFamilyStore(tableNameWithoutKeyspaceName);
+ cfs.indexManager.rebuildIndexesBlocking(Sets.newHashSet(Arrays.asList(indexName)));
+ }
+ );
+ ExecutorService es = Executors.newFixedThreadPool(1);
+ Future future = es.submit(task);
Review Comment:
```suggestion
Future<?> future = es.submit(task);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSANDRA-19564 ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "jaydeepkumar1984 (via GitHub)" <gi...@apache.org>.
jaydeepkumar1984 commented on PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#issuecomment-3434517672
It has been rebased and run a test suite internally. All the tests are passing. I will work on merging this to 4.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "maedhroz (via GitHub)" <gi...@apache.org>.
maedhroz commented on code in PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#discussion_r2427457881
##########
test/distributed/org/apache/cassandra/distributed/test/SecondaryIndexTest.java:
##########
@@ -23,12 +23,20 @@
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
+import java.util.Random;
import java.util.UUID;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
+import com.google.common.collect.Sets;
+
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
Review Comment:
nit: Move cassandra imports down to the proper position
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "maedhroz (via GitHub)" <gi...@apache.org>.
maedhroz commented on code in PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#discussion_r2427513535
##########
test/distributed/org/apache/cassandra/distributed/test/SecondaryIndexTest.java:
##########
@@ -122,4 +132,53 @@ public void test_only_coordinator_chooses_index_for_query()
});
}
}
+
+ @Test
+ public void test_secondary_rebuild_with_small_memtable_memory()
+ {
+ // populate data
+ for (int i = 0 ; i < 100 ; ++i)
+ cluster.coordinator(1).execute(String.format("INSERT INTO %s (k, v) VALUES (?, ?)", tableName), ConsistencyLevel.ALL, i, generateRandomString(50000));
+
+ cluster.forEach(i -> i.flush(KEYSPACE));
+
+ // restart node 1 with small memtable allocation so that index rebuild will cause memtable flush which will need
+ // to reclaim the memory. see CASSANDRA-19564
+ waitOn(cluster.get(1).shutdown());
+ cluster.get(1).config().set("memtable_heap_space", "1MiB");
Review Comment:
If this case runs first, before `test_only_coordinator_chooses_index_for_query`, I'd be a little worried that we'll have overridden `memtable_heap_space` to a non-default value, which could impact subsequent tests. It's not a hot-property, so the restart makes sense, but we might want to ensure the default is active in `@Before`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSANDRA-19564 ReadCommandController should close fast to avoid deadlock when buildi… [cassandra]
Posted by "Runtian (via GitHub)" <gi...@apache.org>.
Runtian commented on code in PR #3700:
URL: https://github.com/apache/cassandra/pull/3700#discussion_r2450390369
##########
test/distributed/org/apache/cassandra/distributed/test/SecondaryIndexTest.java:
##########
@@ -122,4 +132,53 @@ public void test_only_coordinator_chooses_index_for_query()
});
}
}
+
+ @Test
+ public void test_secondary_rebuild_with_small_memtable_memory()
+ {
+ // populate data
+ for (int i = 0 ; i < 100 ; ++i)
+ cluster.coordinator(1).execute(String.format("INSERT INTO %s (k, v) VALUES (?, ?)", tableName), ConsistencyLevel.ALL, i, generateRandomString(50000));
+
+ cluster.forEach(i -> i.flush(KEYSPACE));
+
+ // restart node 1 with small memtable allocation so that index rebuild will cause memtable flush which will need
+ // to reclaim the memory. see CASSANDRA-19564
+ waitOn(cluster.get(1).shutdown());
+ cluster.get(1).config().set("memtable_heap_space", "1MiB");
Review Comment:
Yeah, this is a valid concern, however, this test class starts the nodes in the `BeforeClass` function. I think will just restore the config in the end of the test for the first node.
##########
test/distributed/org/apache/cassandra/distributed/test/SecondaryIndexTest.java:
##########
@@ -23,12 +23,20 @@
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
+import java.util.Random;
import java.util.UUID;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
+import com.google.common.collect.Sets;
+
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.Keyspace;
Review Comment:
updated
##########
test/distributed/org/apache/cassandra/distributed/test/SecondaryIndexTest.java:
##########
@@ -122,4 +132,53 @@ public void test_only_coordinator_chooses_index_for_query()
});
}
}
+
+ @Test
+ public void test_secondary_rebuild_with_small_memtable_memory()
+ {
+ // populate data
+ for (int i = 0 ; i < 100 ; ++i)
+ cluster.coordinator(1).execute(String.format("INSERT INTO %s (k, v) VALUES (?, ?)", tableName), ConsistencyLevel.ALL, i, generateRandomString(50000));
Review Comment:
added a function to support random string with length.
##########
test/distributed/org/apache/cassandra/distributed/test/SecondaryIndexTest.java:
##########
@@ -122,4 +132,53 @@ public void test_only_coordinator_chooses_index_for_query()
});
}
}
+
+ @Test
+ public void test_secondary_rebuild_with_small_memtable_memory()
+ {
+ // populate data
+ for (int i = 0 ; i < 100 ; ++i)
+ cluster.coordinator(1).execute(String.format("INSERT INTO %s (k, v) VALUES (?, ?)", tableName), ConsistencyLevel.ALL, i, generateRandomString(50000));
+
+ cluster.forEach(i -> i.flush(KEYSPACE));
+
+ // restart node 1 with small memtable allocation so that index rebuild will cause memtable flush which will need
+ // to reclaim the memory. see CASSANDRA-19564
+ waitOn(cluster.get(1).shutdown());
+ cluster.get(1).config().set("memtable_heap_space", "1MiB");
+ cluster.get(1).startup();
+ String tableNameWithoutKeyspaceName = tableName.split("\\.")[1];
+ String indexName = String.format("v_index_%d", seq.get());
+ Runnable task = cluster.get(1).runsOnInstance(
+ () -> {
+ ColumnFamilyStore cfs = Keyspace.open(KEYSPACE).getColumnFamilyStore(tableNameWithoutKeyspaceName);
+ cfs.indexManager.rebuildIndexesBlocking(Sets.newHashSet(Arrays.asList(indexName)));
+ }
+ );
+ ExecutorService es = Executors.newFixedThreadPool(1);
+ Future future = es.submit(task);
Review Comment:
updated
##########
src/java/org/apache/cassandra/index/SecondaryIndexManager.java:
##########
@@ -33,6 +33,8 @@
import com.google.common.collect.*;
import com.google.common.primitives.Longs;
import com.google.common.util.concurrent.FutureCallback;
+
+import org.apache.cassandra.db.partitions.ImmutableBTreePartition;
Review Comment:
updated
##########
src/java/org/apache/cassandra/index/SecondaryIndexManager.java:
##########
@@ -918,14 +920,21 @@ public void indexPartition(DecoratedKey key, Set<Index> indexes, int pageSize, R
SinglePartitionPager pager = new SinglePartitionPager(cmd, null, ProtocolVersion.CURRENT);
while (!pager.isExhausted())
{
+ UnfilteredRowIterator partition;
try (ReadExecutionController controller = cmd.executionController();
- WriteContext ctx = keyspace.getWriteHandler().createContextForIndexing();
UnfilteredPartitionIterator page = pager.fetchPageUnfiltered(baseCfs.metadata(), pageSize, controller))
{
if (!page.hasNext())
break;
- try (UnfilteredRowIterator partition = page.next())
+ try (UnfilteredRowIterator onePartition = page.next())
+ {
+ partition = ImmutableBTreePartition.create(onePartition).unfilteredIterator();
+ }
+ }
+
+ try (WriteContext ctx = keyspace.getWriteHandler().createContextForIndexing())
Review Comment:
added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org