You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/05/30 00:25:50 UTC
[GitHub] [pulsar] merlimat opened a new pull request #7110: Don't run synchronous I/O in cache callback
merlimat opened a new pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110
### Motivation
PersistentTopic registers a listener for updates to the policies of
the topic.
PersistentTopic#initializeDispatchRateLimiterIfNeeded runs under a
lock on the dispatchRateLimiter object. This method is called from the
constructor and also from the a listener callback. If the listener
callback triggers during the initial read, then the callbacks will
wait on the lock. This occupies one ForkJoinPool.commonPool
thread. Multiple triggers on the callbacks will occupy multiple
ForkJoinPool.commonPool threads. There needs to be a free thread to
complete the initial read, so this read eventually times out.
This change fixes this in two ways.
Firstly it moves reading from the ZK cache outside of the lock. This
has a knock-on effect in other parts of the code, as
DispatchRateLimiter and SubscriptionRateLimiter no longer read the
Policies for themselves, but require the policies be passed in.
Secondly, it makes the zookeeper cache use its own executor rather
than ForkJoinPool.commonPool. This can help avoid deadlocks but
cannot eliminate them completely if we allow synchronous calls
from within callbacks.
If we allow this kind of call, we are susceptible to
deadlocks, no matter how many threads we add. To allow N threads to
call synchronous calls in callbacks we need N+1 threads. Fixing this
properly would involve ensuring that sychronous calls do not depend on
the async callback executor, which seems like a fairly big change.
This problem was the root cause of some flakes on ReplicatorTest.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-638591050
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
sijie commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-637210288
@merlimat can you rebase the pull request?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-639208904
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-638869357
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-638649278
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
jiazhai commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-638837834
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-639224346
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-639455841
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-639300958
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat closed pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
merlimat closed pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
merlimat commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-843787583
Moving to 2.9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-639947924
@merlimat Looks the failed tests are related to this PR, can we move it to 2.7.0 or release at 2.6.1?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-639233271
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-638787103
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-638681569
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
sijie commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-652786706
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
jiazhai commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-648527392
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-640406005
move to 2.7.0 first.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-638630097
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #7110: Don't run synchronous I/O in cache callback
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #7110:
URL: https://github.com/apache/pulsar/pull/7110#issuecomment-843786197
@merlimat Could you please resolve the conflicts? so that we can onboard this fix in 2.8.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org