You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "David Capwell (Jira)" <ji...@apache.org> on 2020/02/15 00:40:00 UTC
[jira] [Commented] (CASSANDRA-15388) Add compaction allocation
measurement test to support compaction gc optimization.
[ https://issues.apache.org/jira/browse/CASSANDRA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17037376#comment-17037376 ]
David Capwell commented on CASSANDRA-15388:
-------------------------------------------
Sadly com.google.monitoring.runtime.instrumentation.AllocationRecorder#getInstrumentation is packaged protected, would be great to use that to see if the agent is loaded.
To make make things clear, https://github.com/apache/cassandra/compare/trunk...bdeggleston:15388#diff-5d25f445e825fbeef166372877b810d1R140. it would be good if you did
{code}
diff --git a/test/memory/org/apache/cassandra/db/compaction/CompactionAllocationTest.java b/test/memory/org/apache/cassandra/db/compaction/CompactionAllocationTest.java
index 6394b92e6d..53d11c9031 100644
--- a/test/memory/org/apache/cassandra/db/compaction/CompactionAllocationTest.java
+++ b/test/memory/org/apache/cassandra/db/compaction/CompactionAllocationTest.java
@@ -74,7 +74,7 @@ public class CompactionAllocationTest
private static final Logger logger = LoggerFactory.getLogger(CompactionAllocationTest.class);
private static final ThreadMXBean threadMX = (ThreadMXBean) ManagementFactory.getThreadMXBean();
- private static final boolean AGENT_MEASUREMENT = true;
+ private static final boolean AGENT_MEASUREMENT = isAgentLoaded();
private static final boolean PROFILING_READS = false;
private static final boolean PROFILING_COMPACTION = false;
@@ -153,6 +153,14 @@ public class CompactionAllocationTest
}
}
+ private static boolean isAgentLoaded()
+ {
+ AgentMeasurement measurement = new AgentMeasurement();
+ measurement.start();
+ measurement.stop();
+ return measurement.objectsAllocated != 0 || measurement.bytesAllocated != 0;
+ }
+
@BeforeClass
public static void setupClass() throws Throwable
{
{code}
This way we only use the agent if loaded. Right now you don't modify the IDE files, so if you run the test in any IDE it will always log that you allocated 0 bytes; with the patch I posted it uses the thread mbean.
https://github.com/apache/cassandra/compare/trunk...bdeggleston:15388#diff-5d25f445e825fbeef166372877b810d1R79 and https://github.com/apache/cassandra/compare/trunk...bdeggleston:15388#diff-5d25f445e825fbeef166372877b810d1R80. Make this overridable; a system properly would be fine
https://github.com/apache/cassandra/compare/trunk...bdeggleston:15388#diff-5d25f445e825fbeef166372877b810d1R396. remove the branch or get rid of "|| true"
https://github.com/apache/cassandra/compare/trunk...bdeggleston:15388#diff-5d25f445e825fbeef166372877b810d1R402. Why all the sleeps? If I remove them I don't really see a difference
Without:
{code}
INFO [main] 2020-02-14 16:24:20,357 CompactionAllocationTest.java:433 - *** tinyNonOverlapping3 reads summary
INFO [main] 2020-02-14 16:24:20,357 CompactionAllocationTest.java:434 - 11592072 bytes, 4293 /read, 116490000 cpu
INFO [main] 2020-02-14 16:24:20,357 CompactionAllocationTest.java:435 - *** tinyNonOverlapping3 compaction summary
INFO [main] 2020-02-14 16:24:20,357 CompactionAllocationTest.java:436 - 10893688 bytes, 4034 /partition, 4034 /row, 93812000 cpu
{code}
With:
{code}
INFO [main] 2020-02-14 16:25:17,491 CompactionAllocationTest.java:433 - *** tinyNonOverlapping3 reads summary
INFO [main] 2020-02-14 16:25:17,491 CompactionAllocationTest.java:434 - 11589808 bytes, 4292 /read, 109551000 cpu
INFO [main] 2020-02-14 16:25:17,491 CompactionAllocationTest.java:435 - *** tinyNonOverlapping3 compaction summary
INFO [main] 2020-02-14 16:25:17,491 CompactionAllocationTest.java:436 - 10893848 bytes, 4034 /partition, 4034 /row, 74584000 cpu
{code}
The only real change I see is in terms of CPU, which makes sense since you just track total time (though implemented from mbean).
The only hint I have as to why is the comment "// maybe log entries will stop disappearing?" which makes me think logger?
https://github.com/apache/cassandra/compare/trunk...bdeggleston:15388#diff-5d25f445e825fbeef166372877b810d1R200
Looks like you are trying to create a report, but targeting humans? Can you move the strings out of the logger and into something like StringBuilder? Once you complete you can add to the logger for now (though more value if you generate a test report, but fine if this is a different JIRA)
Last comment, it would be nice if you asserted the bounds of memory allocated rather than log.
> Add compaction allocation measurement test to support compaction gc optimization.
> ----------------------------------------------------------------------------------
>
> Key: CASSANDRA-15388
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15388
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local/Compaction
> Reporter: Blake Eggleston
> Assignee: Blake Eggleston
> Priority: Normal
> Fix For: 4.0
>
>
> This adds a test that is able to quickly and accurately measure the effect of potential gc optimizations against a wide range of (synthetic) compaction workloads. This test accurately measures allocation rates from 16 workloads in less that 2 minutes.
> This test uses google’s {{java-allocation-instrumenter}} agent to measure the workloads. Measurements using this agent are very accurate and pretty repeatable from run to run, with most variance being negligible (1-2 bytes per partition), although workloads with larger but fewer partitions vary a bit more (still less that 0.03%).
> The thinking behind this patch is that with compaction, we’re generally interested in the memory allocated per partition, since garbage scales more or less linearly with the number of partitions compacted. So measuring allocation from a small number of partitions that otherwise represent real world use cases is a good enough approximation.
> In addition to helping with compaction optimizations, this test could be used as a template for future optimization work. This pattern could also be used to set allocation limits on workloads/operations and fail CI if the allocation behavior changes past some threshold.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org