You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/11/24 18:31:56 UTC
[GitHub] [pulsar] jerrypeng opened a new pull request #8687: Add timeout to hasMessageAvailable to leader election process in Pulsar Functions
jerrypeng opened a new pull request #8687:
URL: https://github.com/apache/pulsar/pull/8687
### Motivation
The API "hasMessageAvailable" stalls sometimes which leads to the leader initialization process in Pulsar Functions to stall and not complete.
### Modifications
Add a timeout so that a leader does not get stuck
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on a change in pull request #8687: Add timeout to hasMessageAvailable to leader election process in Pulsar Functions
Posted by GitBox <gi...@apache.org>.
merlimat commented on a change in pull request #8687:
URL: https://github.com/apache/pulsar/pull/8687#discussion_r530513211
##########
File path: pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionAssignmentTailer.java
##########
@@ -145,7 +145,7 @@ private Thread getTailerThread() {
try {
Message<byte[]> msg = reader.readNext(1, TimeUnit.SECONDS);
if (msg == null) {
- if (exitOnEndOfTopic && !reader.hasMessageAvailable()) {
+ if (exitOnEndOfTopic && !reader.hasMessageAvailableAsync().get(10, TimeUnit.SECONDS)) {
Review comment:
It's dangerous to have a timeout here, because we might miss messages. Instead we should rely on the timeout in the `GetLastMessageId` RPC (which should be already there now)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jerrypeng commented on a change in pull request #8687: Add timeout to hasMessageAvailable to leader election process in Pulsar Functions
Posted by GitBox <gi...@apache.org>.
jerrypeng commented on a change in pull request #8687:
URL: https://github.com/apache/pulsar/pull/8687#discussion_r530689243
##########
File path: pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionAssignmentTailer.java
##########
@@ -145,7 +145,7 @@ private Thread getTailerThread() {
try {
Message<byte[]> msg = reader.readNext(1, TimeUnit.SECONDS);
if (msg == null) {
- if (exitOnEndOfTopic && !reader.hasMessageAvailable()) {
+ if (exitOnEndOfTopic && !reader.hasMessageAvailableAsync().get(10, TimeUnit.SECONDS)) {
Review comment:
When the timeout expires, a TimeoutException will be thrown and which will force the worker to shutdown and restart. Another worker can more process to becoming leader.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat merged pull request #8687: Add timeout to hasMessageAvailable to leader election process in Pulsar Functions
Posted by GitBox <gi...@apache.org>.
merlimat merged pull request #8687:
URL: https://github.com/apache/pulsar/pull/8687
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org