You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "see-quick (via GitHub)" <gi...@apache.org> on 2026/04/01 08:36:00 UTC
[PR] KAFKA-20378: Make container memory limit configurable [kafka]
see-quick opened a new pull request, #21924:
URL: https://github.com/apache/kafka/pull/21924
This PR adds a possibility to configure memory limit within system tests.
Tried tested:
1. ./tests/docker/ducker-ak up --help
2. DUCKER_RUN_MEMORY=4000m ./tests/docker/ducker-ak up -n 2 --force
3. ./tests/docker/ducker-ak up -n 2 --force --memory 8000m
4. DUCKER_RUN_MEMORY=4000m ./tests/docker/ducker-ak up -n 2 --force --memory 8000m
5. DUCKER_RUN_MEMORY=4000m ./tests/docker/run_tests.sh
6. ./tests/docker/run_tests.sh (points to default)
7. ./tests/docker/ducker-ak up (points to default)
and then also verified:
```bash
podman inspect ducker01 --format '{{.HostConfig.Memory}}'
```
Everything looks good and without issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20378: Make container memory limit configurable [kafka]
Posted by "see-quick (via GitHub)" <gi...@apache.org>.
see-quick commented on PR #21924:
URL: https://github.com/apache/kafka/pull/21924#issuecomment-4204236483
> @see-quick Just out of curiosity, have you seen any E2E failures due to OOM?
I saw OOM issues but were primarily caused by differences in OS. In other words on Debian in went well but CentOS had a few problems. One particular set of tests if I remember correctly were a bit flaky due to need of resources (i.e., replication_test.py).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20378: Make container memory limit configurable [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 merged PR #21924:
URL: https://github.com/apache/kafka/pull/21924
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20378: Make container memory limit configurable [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #21924:
URL: https://github.com/apache/kafka/pull/21924#issuecomment-4199270120
@see-quick Just out of curiosity, have you seen any E2E failures due to OOM?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20378: Make container memory limit configurable [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #21924:
URL: https://github.com/apache/kafka/pull/21924#issuecomment-4204353161
> I saw OOM issues but were primarily caused by differences in OS. In other words on Debian in went well but CentOS had a few problems. One particular set of tests if I remember correctly were a bit flaky due to need of resources (i.e., replication_test.py).
Thanks for the explanation! I'd love to learn more about this: what makes CentOS more prone to OOM in these tests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20378: Make container memory limit configurable [kafka]
Posted by "see-quick (via GitHub)" <gi...@apache.org>.
see-quick commented on PR #21924:
URL: https://github.com/apache/kafka/pull/21924#issuecomment-4222776393
> > I saw OOM issues but were primarily caused by differences in OS. In other words on Debian in went well but CentOS had a few problems. One particular set of tests if I remember correctly were a bit flaky due to need of resources (i.e., replication_test.py).
>
> Thanks for the explanation! I'd love to learn more about this: what makes CentOS more prone to OOM in these tests?
I didn't do much investigation around this (back then) but I know it was mostly related on older versions of CentOS (i.e., 7 or less) and its Kernel where I think there were some differences around KMEM. I wanted to avoid doing fine-tuning on OS-level (i.e., modifying default kernel options) so I just increased a memory a it worked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20378: Make container memory limit configurable [kafka]
Posted by "see-quick (via GitHub)" <gi...@apache.org>.
see-quick commented on PR #21924:
URL: https://github.com/apache/kafka/pull/21924#issuecomment-4198381520
Can anyone check please @m1a2st @mimaison @chia7712 ^^? Thanks in advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20378: Make container memory limit configurable [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21924:
URL: https://github.com/apache/kafka/pull/21924#issuecomment-4211367468
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the `triage` label
should be removed to prevent this automation from happening again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org