You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Ekaterina Dimitrova (Jira)" <ji...@apache.org> on 2020/10/27 21:08:00 UTC
[jira] [Comment Edited] (CASSANDRA-16228) Fix race condition on
ColumnFamilyStore and TableMetrics initialization that makes MetricListener
to see an instance in an unknown state
[ https://issues.apache.org/jira/browse/CASSANDRA-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221755#comment-17221755 ]
Ekaterina Dimitrova edited comment on CASSANDRA-16228 at 10/27/20, 9:07 PM:
----------------------------------------------------------------------------
The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition.
In 4.0, as part of CASSANDRA-, requests implementation in CacheMetrics was changed but this change (2n suggestion here ) prevents us from using the codadhale [addListener() |https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-, changing its type from Metered to Meter and using the mark method.
The race condition was fixed itself in 3.0, 3.11, 4.0.
[4.0 pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/]
[3.0 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/]
[3.11 branch |https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI |https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes]
[3.0 patch |https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after* applying first this [patch |https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0
was (Author: e.dimitrova):
The patch adds a [test|https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R122] to cover the mentioned race condition.
In 4.0, as part of CASSANDRA-, requests implementation in CacheMetrics was changed but this change (2n suggestion [here | https://issues.apache.org/jira/browse/CASSANDRA-14626?focusedCommentId=16587227&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16587227]) prevents us from using the codadhale [addListener() | https://github.com/ekaterinadimitrova2/cassandra/commit/612d71cd09967e725770895ebfcf528f11353bd4#diff-3b7425b96625eda53b205ad149edea919df71b29ef01efc602d8ce28b86f3f59R136]. That is why I reverted the requests implementation to the original [version | https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] from CASSANDRA-, changing its type from Metered to Meter and using the mark method.
The race condition was fixed itself in 3.0, 3.11, 4.0.
[4.0 pull request | https://github.com/ekaterinadimitrova2/cassandra/pull/67] | [JAVA 8 CI | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/8df700c5-1775-433f-8dfc-3b3391f3d0ae] | [JAVA 11 CI | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/450/workflows/4f8946b5-6448-4dd6-b231-8f288c861f34] [Jenkins CI | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/144/]
[3.0 branch | https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] | [CI | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/145/]
[3.11 branch | https://github.com/ekaterinadimitrova2/cassandra/commit/dfcb3e82e952f8b1dbeb1fe91a3ed3fcb07abc5e] | [CI | https://jenkins-cm4.apache.org/job/Cassandra-devbranch/146/changes]
[3.0 patch | https://github.com/ekaterinadimitrova2/cassandra/commit/898880b5f6d7635cddc4eec1f19513fe61de5b43] to be merged also to 3.11 and 4.0 *after * applying first this [patch | https://github.com/ekaterinadimitrova2/cassandra/commit/c66e6641a5a5f7e9241212a8193feb6a66771d3f] to 4.0
> Fix race condition on ColumnFamilyStore and TableMetrics initialization that makes MetricListener to see an instance in an unknown state
> ----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-16228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16228
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
> Reporter: Ekaterina Dimitrova
> Assignee: Ekaterina Dimitrova
> Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> There is a dependency between TableMetrics, SecondaryIndexManager and Tracker which requires certain order of the operations in ColumnFamilyStore constructor.
> The following test will be added to ColumnFamilyMetricTest to show the issue:
> {code:java}
> @Test
> public void testStartupRaceConditionOnMetricListeners()
> {
> // Since the ColumnFamilyStore instance reference escapes during the construction
> // we have a race condition and listeners can see an instance that is in an unknown state.
> // This test just check that all callbacks can access the data without throwing any exception.
> registerMetricListener();
> SchemaLoader.createKeyspace("Keyspace2",
> KeyspaceParams.simple(1),
> SchemaLoader.standardCFMD("Keyspace2", "Standard2"));
> }
> private void registerMetricListener()
> {
> CassandraMetricsRegistry.Metrics.addListener(new MetricRegistryListener.Base()
> {
> @Override
> public void onGaugeAdded(String name, Gauge<?> gauge)
> {
> gauge.getValue();
> }
> @Override
> public void onGaugeRemoved(String name)
> {
> }
> @Override
> public void onCounterAdded(String name, Counter counter)
> {
> counter.getCount();
> }
> @Override
> public void onCounterRemoved(String name)
> {
> }
> @Override
> public void onHistogramAdded(String name, Histogram histogram)
> {
> histogram.getCount();
> }
> @Override
> public void onHistogramRemoved(String name)
> {
> }
> @Override
> public void onMeterAdded(String name, Meter meter)
> {
> meter.getCount();
> }
> @Override
> public void onMeterRemoved(String name)
> {
> }
> @Override
> public void onTimerAdded(String name, Timer timer)
> {
> timer.getCount();
> }
> @Override
> public void onTimerRemoved(String name)
> {
> }
> });{code}
> *NOTE:* In order to be able to run the test and use registerMetricListener() method a small modification to requests in CacheMetrics is needed. This will be submitted and further explained with a coming patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org