You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by "skoppu22 (via GitHub)" <gi...@apache.org> on 2026/04/09 15:34:42 UTC
[PR] CASSSIDECAR-423 Fix circle CI pipelines OOM [cassandra-sidecar]
skoppu22 opened a new pull request, #334:
URL: https://github.com/apache/cassandra-sidecar/pull/334
Tune parallelism and heap in Circle CI config. Also mark some tests as heavy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSSIDECAR-423 Fix circle CI pipelines OOM [cassandra-sidecar]
Posted by "skoppu22 (via GitHub)" <gi...@apache.org>.
skoppu22 commented on PR #334:
URL: https://github.com/apache/cassandra-sidecar/pull/334#issuecomment-4216402083
CI pipeline https://rio.apple.com/projects/ase-skoppu-cassandra-analytics/pipeline-specs/ase-skoppu-cassandra-analytics-prb-trunk/pipelines/a32d50f0-47cc-4e66-a5ad-42698f8226c4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSSIDECAR-423 Fix circle CI pipelines OOM [cassandra-sidecar]
Posted by "sarankk (via GitHub)" <gi...@apache.org>.
sarankk commented on code in PR #334:
URL: https://github.com/apache/cassandra-sidecar/pull/334#discussion_r3059977924
##########
.circleci/config.yml:
##########
@@ -119,16 +119,16 @@ jobs:
docker:
- image: cimg/openjdk:11.0
environment:
- INTEGRATION_MAX_PARALLEL_FORKS: 3
- INTEGRATION_MAX_HEAP_SIZE: "2500M"
+ INTEGRATION_MAX_PARALLEL_FORKS: 2
+ INTEGRATION_MAX_HEAP_SIZE: "2000M"
resource_class: large
steps:
- install_common
- checkout
- attach_workspace:
at: dtest-jars
- run: ./scripts/install-shaded-dtest-jar-local.sh
- - run: ./gradlew --no-daemon -PdtestVersion=4.0.16 -Dcassandra.sidecar.versions_to_test="4.0" checkstyleIntegrationTest integrationTestLightWeight --stacktrace
+ - run: ./gradlew --no-daemon --max-workers=2 -PdtestVersion=4.0.16 -Dcassandra.sidecar.versions_to_test="4.0" checkstyleIntegrationTest integrationTestLightWeight --stacktrace
Review Comment:
Thanks @skoppu22 for the fix, could you also add a comment, why this setting is important, will be good for referring back.
##########
.circleci/config.yml:
##########
@@ -119,16 +119,16 @@ jobs:
docker:
- image: cimg/openjdk:11.0
environment:
- INTEGRATION_MAX_PARALLEL_FORKS: 3
- INTEGRATION_MAX_HEAP_SIZE: "2500M"
+ INTEGRATION_MAX_PARALLEL_FORKS: 2
+ INTEGRATION_MAX_HEAP_SIZE: "2000M"
Review Comment:
Nice, setting `INTEGRATION_MAX_PARALLEL_FORKS: 2` makes sense since we are setting `--max-workers=2`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSSIDECAR-423 Fix circle CI pipelines OOM [cassandra-sidecar]
Posted by "skoppu22 (via GitHub)" <gi...@apache.org>.
skoppu22 commented on code in PR #334:
URL: https://github.com/apache/cassandra-sidecar/pull/334#discussion_r3059827649
##########
.circleci/config.yml:
##########
@@ -119,16 +119,16 @@ jobs:
docker:
- image: cimg/openjdk:11.0
environment:
- INTEGRATION_MAX_PARALLEL_FORKS: 3
- INTEGRATION_MAX_HEAP_SIZE: "2500M"
+ INTEGRATION_MAX_PARALLEL_FORKS: 2
+ INTEGRATION_MAX_HEAP_SIZE: "2000M"
Review Comment:
Light weight tests don't need 2500M heap, they can run just fine with lower heap as well. The main fix is --max-workers and reducing the parallelism. Reduced max heap value to ensure we don't go beyond calculations and hit OOM again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSSIDECAR-423 Fix circle CI pipelines OOM [cassandra-sidecar]
Posted by "skoppu22 (via GitHub)" <gi...@apache.org>.
skoppu22 commented on PR #334:
URL: https://github.com/apache/cassandra-sidecar/pull/334#issuecomment-4216479435
Circle CI pipeline https://app.circleci.com/pipelines/github/skoppu22/cassandra-sidecar/51/workflows/74f005a2-c869-45a3-83f4-7e6c8f671a60
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSSIDECAR-423 Fix circle CI pipelines OOM [cassandra-sidecar]
Posted by "jyothsnakonisa (via GitHub)" <gi...@apache.org>.
jyothsnakonisa commented on code in PR #334:
URL: https://github.com/apache/cassandra-sidecar/pull/334#discussion_r3059800297
##########
.circleci/config.yml:
##########
@@ -119,16 +119,16 @@ jobs:
docker:
- image: cimg/openjdk:11.0
environment:
- INTEGRATION_MAX_PARALLEL_FORKS: 3
- INTEGRATION_MAX_HEAP_SIZE: "2500M"
+ INTEGRATION_MAX_PARALLEL_FORKS: 2
+ INTEGRATION_MAX_HEAP_SIZE: "2000M"
Review Comment:
Curious how is reducing INTEGRATION_MAX_HEAP_SIZE from "2500M" to "2000M" fixes the CI
##########
server/src/test/integration/org/apache/cassandra/sidecar/cluster/driver/SidecarLoadBalancingPolicyTest.java:
##########
@@ -42,6 +43,7 @@
/**
* A test for the SidecarLoadBalancingPolicy
*/
+@Tag("heavy")
Review Comment:
How did you find out that only these tests have to be marked as heavy to resolve CI issues
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSSIDECAR-423 Fix circle CI pipelines OOM [cassandra-sidecar]
Posted by "skoppu22 (via GitHub)" <gi...@apache.org>.
skoppu22 commented on code in PR #334:
URL: https://github.com/apache/cassandra-sidecar/pull/334#discussion_r3059833265
##########
server/src/test/integration/org/apache/cassandra/sidecar/cluster/driver/SidecarLoadBalancingPolicyTest.java:
##########
@@ -42,6 +43,7 @@
/**
* A test for the SidecarLoadBalancingPolicy
*/
+@Tag("heavy")
Review Comment:
1. High node count (5-10) — each in-JVM Cassandra instance holds its own heap, memtables,
caches, etc.
2. network=true — enables full network simulation in the dtest framework, substantially
increasing memory per node (used by Joining, Moving, Replacement tests)
3. Dynamic node operations — bootstrap/move/replace require ByteBuddy instrumentation and
maintaining complex state transitions across multiple node lifecycles
When INTEGRATION_MAX_PARALLEL_FORKS=2, multiple 5-10 node tests could run
simultaneously, easily exceeding the CI container's memory. Moving them to the separate
heavyweight job ensures they run in isolation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSSIDECAR-423 Fix circle CI pipelines OOM [cassandra-sidecar]
Posted by "jyothsnakonisa (via GitHub)" <gi...@apache.org>.
jyothsnakonisa merged PR #334:
URL: https://github.com/apache/cassandra-sidecar/pull/334
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org