You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by GitBox <gi...@apache.org> on 2021/07/27 17:32:30 UTC
[GitHub] [samza] lakshmi-manasa-g opened a new pull request #1511: Samza-2665: AzureBlob SystemProducer: stop block upload retrying when InterruptedException is thrown
lakshmi-manasa-g opened a new pull request #1511:
URL: https://github.com/apache/samza/pull/1511
Symptom: interrupted thread stays stuck for a long time.
Cause: retry several times for all exceptions during block upload.
Changes: for interrupted exception, stop retry and throw exception that will be bubbled up.
Tests: added unit test
API changes: None
Usage/upgrade instructions: N/A
Backwards compatible: yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [samza] rmatharu commented on a change in pull request #1511: Samza-2665: AzureBlob SystemProducer: stop block upload retrying when InterruptedException is thrown
Posted by GitBox <gi...@apache.org>.
rmatharu commented on a change in pull request #1511:
URL: https://github.com/apache/samza/pull/1511#discussion_r677864641
##########
File path: samza-azure/src/main/java/org/apache/samza/system/azureblob/avro/AzureBlobOutputStream.java
##########
@@ -335,6 +335,11 @@ public void run() {
// StageBlock generates exception on Failure.
stageBlock(blockIdEncoded, outputStream, blockSize);
break;
+ } catch (InterruptedException e) {
+ String msg = "Upload block for blob: " + blobAsyncClient.getBlobUrl().toString()
+ + " failed for blockid: " + blockId + " due to InterruptedException ";
+ LOG.error(msg, e);
+ throw new AzureException("InterruptedException encountered during block upload. Will not retry.", e);
Review comment:
Is there a reason why on a thread being interrupted should it not be retried?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [samza] rmatharu merged pull request #1511: Samza-2665: AzureBlob SystemProducer: stop block upload retrying when InterruptedException is thrown
Posted by GitBox <gi...@apache.org>.
rmatharu merged pull request #1511:
URL: https://github.com/apache/samza/pull/1511
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [samza] rmatharu commented on a change in pull request #1511: Samza-2665: AzureBlob SystemProducer: stop block upload retrying when InterruptedException is thrown
Posted by GitBox <gi...@apache.org>.
rmatharu commented on a change in pull request #1511:
URL: https://github.com/apache/samza/pull/1511#discussion_r677864384
##########
File path: samza-azure/src/main/java/org/apache/samza/system/azureblob/avro/AzureBlobOutputStream.java
##########
@@ -335,6 +335,11 @@ public void run() {
// StageBlock generates exception on Failure.
stageBlock(blockIdEncoded, outputStream, blockSize);
break;
+ } catch (InterruptedException e) {
+ String msg = "Upload block for blob: " + blobAsyncClient.getBlobUrl().toString()
Review comment:
nit: String.format use
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [samza] rmatharu commented on pull request #1511: Samza-2665: AzureBlob SystemProducer: stop block upload retrying when InterruptedException is thrown
Posted by GitBox <gi...@apache.org>.
rmatharu commented on pull request #1511:
URL: https://github.com/apache/samza/pull/1511#issuecomment-889506737
makes sense,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [samza] lakshmi-manasa-g commented on a change in pull request #1511: Samza-2665: AzureBlob SystemProducer: stop block upload retrying when InterruptedException is thrown
Posted by GitBox <gi...@apache.org>.
lakshmi-manasa-g commented on a change in pull request #1511:
URL: https://github.com/apache/samza/pull/1511#discussion_r677886094
##########
File path: samza-azure/src/main/java/org/apache/samza/system/azureblob/avro/AzureBlobOutputStream.java
##########
@@ -335,6 +335,11 @@ public void run() {
// StageBlock generates exception on Failure.
stageBlock(blockIdEncoded, outputStream, blockSize);
break;
+ } catch (InterruptedException e) {
+ String msg = "Upload block for blob: " + blobAsyncClient.getBlobUrl().toString()
+ + " failed for blockid: " + blockId + " due to InterruptedException ";
+ LOG.error(msg, e);
+ throw new AzureException("InterruptedException encountered during block upload. Will not retry.", e);
Review comment:
original idea of retry was to deal with azure blob storage connection issues.
Now the user of AzureBlobSystemProducer has a need where they want to interrupt the thread themselves if it doesnt stop within a set period of time (stop on thread invoked by them again). In such a scenario, they expect the thread to be killed immediately and producer bubble up the exception so that they can handle the exception in an appropriate manner. since they are dealing at the thread level and not producer level, it becomes hard for them to identify the producer of the thread and handle producer exceptions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org