You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "jyothsnakonisa (via GitHub)" <gi...@apache.org> on 2026/03/30 21:27:42 UTC
[PR] CASSANALYTICS-142 : SidecarCdcClient should be passed as a constructor parameter for SidecarCdc to avoid thread/resource leaks [cassandra-analytics]
jyothsnakonisa opened a new pull request, #188:
URL: https://github.com/apache/cassandra-analytics/pull/188
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org
Re: [PR] CASSANALYTICS-142 : SidecarCdcClient should be passed as a constructor parameter for SidecarCdc to avoid thread/resource leaks [cassandra-analytics]
Posted by "jyothsnakonisa (via GitHub)" <gi...@apache.org>.
jyothsnakonisa merged PR #188:
URL: https://github.com/apache/cassandra-analytics/pull/188
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org
Re: [PR] CASSANALYTICS-142 : SidecarCdcClient should be passed as a constructor parameter for SidecarCdc to avoid thread/resource leaks [cassandra-analytics]
Posted by "jyothsnakonisa (via GitHub)" <gi...@apache.org>.
jyothsnakonisa commented on code in PR #188:
URL: https://github.com/apache/cassandra-analytics/pull/188#discussion_r3017366745
##########
cassandra-analytics-cdc-sidecar/src/main/java/org/apache/cassandra/cdc/sidecar/SidecarCdc.java:
##########
@@ -75,6 +75,27 @@ public static SidecarCdcBuilder builder(@NotNull String jobId,
cdcStats);
}
Review Comment:
good catch, this is no longer needed and we should remove it as the builder path creates new SidecarCdcClient.
I also think, we should remove the constructor which creates new SidecarCdcClient in the constructor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org
Re: [PR] CASSANALYTICS-142 : SidecarCdcClient should be passed as a constructor parameter for SidecarCdc to avoid thread/resource leaks [cassandra-analytics]
Posted by "yifan-c (via GitHub)" <gi...@apache.org>.
yifan-c commented on code in PR #188:
URL: https://github.com/apache/cassandra-analytics/pull/188#discussion_r3013609682
##########
cassandra-analytics-cdc-sidecar/src/main/java/org/apache/cassandra/cdc/sidecar/SidecarCdcClient.java:
##########
@@ -52,21 +56,42 @@
import static org.apache.cassandra.spark.utils.Properties.DEFAULT_SIDECAR_PORT;
import static org.apache.cassandra.spark.utils.Properties.DEFAULT_TIMEOUT_SECONDS;
-public class SidecarCdcClient
+public class SidecarCdcClient implements AutoCloseable
{
final ClientConfig config;
final SidecarClient sidecarClient;
final ICdcStats stats;
- public SidecarCdcClient(ClientConfig config,
- SidecarClient sidecarClient,
- ICdcStats stats)
+ public SidecarCdcClient(ClientConfig clientConfig,
+ CdcSidecarInstancesProvider instancesProvider,
+ SecretsProvider secretsProvider,
+ ICdcStats cdcStats) throws IOException
+ {
+ this(clientConfig,
+ Sidecar.from(new SimpleSidecarInstancesProvider(instancesProvider.instances()
+ .stream()
+ .map(i -> new SidecarInstanceImpl(i.hostname(), i.port()))
+ .collect(Collectors.toList())),
+ clientConfig.toGenericSidecarConfig(),
+ secretsProvider),
+ cdcStats);
+ }
+
+ private SidecarCdcClient(ClientConfig config,
+ SidecarClient sidecarClient,
+ ICdcStats stats)
{
this.config = config;
this.sidecarClient = sidecarClient;
this.stats = stats;
}
+ @Override
+ public void close() throws Exception
+ {
+ sidecarClient.close();
+ }
Review Comment:
Please add a detailed comment to explain who closes the shared client and when it is closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org
Re: [PR] CASSANALYTICS-142 : SidecarCdcClient should be passed as a constructor parameter for SidecarCdc to avoid thread/resource leaks [cassandra-analytics]
Posted by "yifan-c (via GitHub)" <gi...@apache.org>.
yifan-c commented on code in PR #188:
URL: https://github.com/apache/cassandra-analytics/pull/188#discussion_r3013600272
##########
cassandra-analytics-cdc-sidecar/src/main/java/org/apache/cassandra/cdc/sidecar/SidecarCdcBuilder.java:
##########
@@ -61,21 +56,11 @@ public class SidecarCdcBuilder extends CdcBuilder
SecretsProvider secretsProvider,
ICdcStats cdcStats) throws IOException
{
- this(
- jobId,
- partitionId,
- cdcOptions,
- clusterConfigProvider,
- eventConsumer,
- schemaSupplier,
- tokenRangeSupplier,
- clientConfig,
- Sidecar.from(new SimpleSidecarInstancesProvider(sidecarInstancesProvider.instances().stream()
- .map(i -> new SidecarInstanceImpl(i.hostname(), i.port()))
- .collect(Collectors.toList())),
- clientConfig.toGenericSidecarConfig(), secretsProvider),
- cdcStats
- );
+ super(jobId, partitionId, eventConsumer, schemaSupplier);
+ this.clusterConfigProvider = clusterConfigProvider;
+ this.sidecarCdcClient = new SidecarCdcClient(clientConfig, sidecarInstancesProvider, secretsProvider, cdcStats);
+ withCdcOptions(cdcOptions);
+ withTokenRangeSupplier(tokenRangeSupplier);
}
Review Comment:
You should be able to reuse the other constructor.
```java
this(jobId, partitionId, cdcOptions, clusterConfigProvider, eventConsumer, schemaSupplier, tokenRangeSupplier,
new SidecarCdcClient(clientConfig, sidecarInstancesProvider, secretsProvider, cdcStats),
cdcStats);
```
However, the real question is: why do we need the constructor that creates `SidecarCdcClient`? We want to share the global `SidecarCdcClient` instance.
##########
cassandra-analytics-cdc-sidecar/src/main/java/org/apache/cassandra/cdc/sidecar/SidecarCdcClient.java:
##########
@@ -52,21 +56,42 @@
import static org.apache.cassandra.spark.utils.Properties.DEFAULT_SIDECAR_PORT;
import static org.apache.cassandra.spark.utils.Properties.DEFAULT_TIMEOUT_SECONDS;
-public class SidecarCdcClient
+public class SidecarCdcClient implements AutoCloseable
{
final ClientConfig config;
final SidecarClient sidecarClient;
final ICdcStats stats;
- public SidecarCdcClient(ClientConfig config,
- SidecarClient sidecarClient,
- ICdcStats stats)
+ public SidecarCdcClient(ClientConfig clientConfig,
+ CdcSidecarInstancesProvider instancesProvider,
+ SecretsProvider secretsProvider,
+ ICdcStats cdcStats) throws IOException
+ {
+ this(clientConfig,
+ Sidecar.from(new SimpleSidecarInstancesProvider(instancesProvider.instances()
+ .stream()
+ .map(i -> new SidecarInstanceImpl(i.hostname(), i.port()))
+ .collect(Collectors.toList())),
+ clientConfig.toGenericSidecarConfig(),
+ secretsProvider),
+ cdcStats);
+ }
+
+ private SidecarCdcClient(ClientConfig config,
+ SidecarClient sidecarClient,
+ ICdcStats stats)
{
this.config = config;
this.sidecarClient = sidecarClient;
this.stats = stats;
}
+ @Override
+ public void close() throws Exception
+ {
+ sidecarClient.close();
+ }
Review Comment:
Please add a detailed comment to explain who closes the client and when it is closed.
##########
cassandra-analytics-cdc-sidecar/src/main/java/org/apache/cassandra/cdc/sidecar/SidecarCdc.java:
##########
@@ -75,6 +75,27 @@ public static SidecarCdcBuilder builder(@NotNull String jobId,
cdcStats);
}
+ public static SidecarCdcBuilder builder(@NotNull String jobId,
Review Comment:
Please add the java doc of the public method. More importantly, please document the expectation on who should manage the lifecycle of `SidecarCdcClient` and make it clear that Cdc/SidecarCdc does not close the client.
##########
cassandra-analytics-cdc-sidecar/src/main/java/org/apache/cassandra/cdc/sidecar/SidecarCdc.java:
##########
@@ -75,6 +75,27 @@ public static SidecarCdcBuilder builder(@NotNull String jobId,
cdcStats);
}
Review Comment:
Do we still need this build? Please remove it if it is no longer needed. One reason for removal is that the builder path creates a new sidecar cdc client.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org
Re: [PR] CASSANALYTICS-142 : SidecarCdcClient should be passed as a constructor parameter for SidecarCdc to avoid thread/resource leaks [cassandra-analytics]
Posted by "jyothsnakonisa (via GitHub)" <gi...@apache.org>.
jyothsnakonisa commented on code in PR #188:
URL: https://github.com/apache/cassandra-analytics/pull/188#discussion_r3017330229
##########
cassandra-analytics-cdc-sidecar/src/main/java/org/apache/cassandra/cdc/sidecar/SidecarCdcBuilder.java:
##########
@@ -61,21 +56,11 @@ public class SidecarCdcBuilder extends CdcBuilder
SecretsProvider secretsProvider,
ICdcStats cdcStats) throws IOException
{
- this(
- jobId,
- partitionId,
- cdcOptions,
- clusterConfigProvider,
- eventConsumer,
- schemaSupplier,
- tokenRangeSupplier,
- clientConfig,
- Sidecar.from(new SimpleSidecarInstancesProvider(sidecarInstancesProvider.instances().stream()
- .map(i -> new SidecarInstanceImpl(i.hostname(), i.port()))
- .collect(Collectors.toList())),
- clientConfig.toGenericSidecarConfig(), secretsProvider),
- cdcStats
- );
+ super(jobId, partitionId, eventConsumer, schemaSupplier);
+ this.clusterConfigProvider = clusterConfigProvider;
+ this.sidecarCdcClient = new SidecarCdcClient(clientConfig, sidecarInstancesProvider, secretsProvider, cdcStats);
+ withCdcOptions(cdcOptions);
+ withTokenRangeSupplier(tokenRangeSupplier);
}
Review Comment:
We want the `SidecarCdcClient` object to be singleton created in the Sidecar process and it should be shared across all the `SidecarCdc` instances which are CDC iterators.
I have added this new constructor to take `sidecarCdcClient` object as parameter so that we can have singleton object and can pass it to all iterators to avoid creating one `sidecarCdcClient` instance per iterator.
If we have the `SidecarCdcClient` creation inside the constructor, then on token range change when new iterator is created a new `SidecarCdcClient` is created, which we want to avoid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org