You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/01/27 23:50:55 UTC
[GitHub] [pulsar] csthomas1 opened a new pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
csthomas1 opened a new pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349
Fixes #9345
### Motivation
When I enable either the Process or Kubernetes function Runtimes and set the parameter "maxPendingAsyncRequests" to a value other than 1000 in the function worker configuration (e.g. functions_worker.yml), any function instances launched by the worker continue to have maxPendingAsyncRequests = 1000. I'd like to be able to adjust this parameter to guard against my function running out of memory -- without increasing the function heap size -- due to too many publish operations in flight, as can occur when the output topic's backlog quota has been reached.
### Modifications
Modified RuntimeUtils.getCmd to add --pending-sync-requests to the list of arguments passed to JavaInstanceStarter
### Verifying this change
This change added tests and can be verified as follows:
- Updated pulsar-functions/runtime/src/test/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntimeTest.java to test for presence of --pending-async-requests argument in generated commandline, and increased expected arg counts by 2 to account for expected presence of --pending-async-requests <value>
- Updated pulsar-functions/runtime/src/test/java/org/apache/pulsar/functions/runtime/process/ProcessRuntimeTest.java to test for presence of --pending-async-requests argument in generated commandline, and increased expected arg counts by 2 to account for expected presence of --pending-async-requests <value>
- Updated pulsar-functions/runtime/src/test/java/org/apache/pulsar/functions/worker/WorkerApiV2ResourceConfigTest.java to verify that the configured value of "maxPendingAsyncRequests" was being properly loaded into the WorkerConfig
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): no
- The public API: no
- The schema: no
- The default values of configurations: no
- The wire protocol: no
- The rest endpoints: no
- The admin cli options: no
- Anything that affects deployment: no
### Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
- If a feature is not applicable for documentation, explain why?
- bug fix to make an existing configuration parameter operational
- If a feature is not documented yet in this PR, please create a followup issue for adding the documentation
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zymap commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
zymap commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-784195037
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-769029870
@nlu90 Please also help review this PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jerrypeng commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
jerrypeng commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-771944428
@csthomas1 thanks for fix!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] csthomas1 removed a comment on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
csthomas1 removed a comment on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-785976540
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] csthomas1 commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
csthomas1 commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-795733309
@zymap @sijie The tests are finally green for this PR...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] csthomas1 removed a comment on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
csthomas1 removed a comment on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-785545125
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] csthomas1 commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
csthomas1 commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-769198159
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] csthomas1 commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
csthomas1 commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-785976540
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jerrypeng commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
jerrypeng commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-771944428
@csthomas1 thanks for fix!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] csthomas1 commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
csthomas1 commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-786402028
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
sijie commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-797857291
@csthomas1 Awesome! Thank you for your contribution!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] csthomas1 commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
csthomas1 commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-785545125
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Renkai commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
Renkai commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-770157091
Please merge the newest master and rerun tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zymap commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
zymap commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-785069698
@csthomas1 Looks like the test failed by this change. Could you please take a look?
move this to the next release.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie merged pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
sijie merged pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-829029259
this patch does not apply to branch-7.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
sijie commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-783201121
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on pull request #9349: issue#9345 -- bug fix for RuntimeUtils.getCmd not setting pending-async-requests
Posted by GitBox <gi...@apache.org>.
sijie commented on pull request #9349:
URL: https://github.com/apache/pulsar/pull/9349#issuecomment-777184997
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org