You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "ajothomas (via GitHub)" <gi...@apache.org> on 2024/10/09 21:06:21 UTC
[PR] Log bob deserialization exception for snapshot blobs [samza]
ajothomas opened a new pull request, #1711:
URL: https://github.com/apache/samza/pull/1711
**Symptom**:
`BlobStoreUtil` doesn't log the blob id today when it fails to deserialize the snapshot blob. Without the blob id, it is tricky to debug why the deserialization issue happened.
**Changes**:
- Log the blob id if we are unable to deserialize the snapshot blobs
**Tests**:
- ./gradlew build
**API Changes**:
None
**Upgrade Instructions**:
None
**Usage Instructions**:
None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Log bob deserialization exception for snapshot blobs [samza]
Posted by "xinyuiscool (via GitHub)" <gi...@apache.org>.
xinyuiscool commented on code in PR #1711:
URL: https://github.com/apache/samza/pull/1711#discussion_r1795867796
##########
samza-core/src/main/java/org/apache/samza/storage/blobstore/util/BlobStoreUtil.java:
##########
@@ -182,7 +182,13 @@ public CompletableFuture<SnapshotIndex> getSnapshotIndex(String blobId, Metadata
return FutureUtil.executeAsyncWithRetries(opName, () -> {
ByteArrayOutputStream indexBlobStream = new ByteArrayOutputStream(); // no need to close ByteArrayOutputStream
return blobStoreManager.get(blobId, indexBlobStream, metadata, getDeleted).toCompletableFuture()
- .thenApplyAsync(f -> snapshotIndexSerde.fromBytes(indexBlobStream.toByteArray()), executor);
+ .thenApplyAsync(f -> snapshotIndexSerde.fromBytes(indexBlobStream.toByteArray()), executor)
+ .handle((snapshotIndex, ex) -> {
+ if (ex != null) {
+ throw new SamzaException(String.format("Unable to deserialize SnapshotIndex bytes for blob ID: %s", blobId), ex);
Review Comment:
Seems we might get other kinds of exceptions here, e.g. GET exceptions, so change the log message to be "Unable to get SnapshotIndex ..." ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Log bob deserialization exception for snapshot blobs [samza]
Posted by "ajothomas (via GitHub)" <gi...@apache.org>.
ajothomas commented on code in PR #1711:
URL: https://github.com/apache/samza/pull/1711#discussion_r1794310116
##########
samza-core/src/main/java/org/apache/samza/storage/blobstore/util/BlobStoreUtil.java:
##########
@@ -182,7 +182,13 @@ public CompletableFuture<SnapshotIndex> getSnapshotIndex(String blobId, Metadata
return FutureUtil.executeAsyncWithRetries(opName, () -> {
ByteArrayOutputStream indexBlobStream = new ByteArrayOutputStream(); // no need to close ByteArrayOutputStream
return blobStoreManager.get(blobId, indexBlobStream, metadata, getDeleted).toCompletableFuture()
- .thenApplyAsync(f -> snapshotIndexSerde.fromBytes(indexBlobStream.toByteArray()), executor);
+ .thenApplyAsync(f -> {
+ try {
+ return snapshotIndexSerde.fromBytes(indexBlobStream.toByteArray());
+ } catch (Exception e) {
+ throw new SamzaException(String.format("Unable to deserialize SnapshotIndex bytes for blob ID: %s", blobId), e);
Review Comment:
Done. thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Log bob deserialization exception for snapshot blobs [samza]
Posted by "ajothomas (via GitHub)" <gi...@apache.org>.
ajothomas merged PR #1711:
URL: https://github.com/apache/samza/pull/1711
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Log bob deserialization exception for snapshot blobs [samza]
Posted by "dxichen (via GitHub)" <gi...@apache.org>.
dxichen commented on code in PR #1711:
URL: https://github.com/apache/samza/pull/1711#discussion_r1794286044
##########
samza-core/src/main/java/org/apache/samza/storage/blobstore/util/BlobStoreUtil.java:
##########
@@ -182,7 +182,13 @@ public CompletableFuture<SnapshotIndex> getSnapshotIndex(String blobId, Metadata
return FutureUtil.executeAsyncWithRetries(opName, () -> {
ByteArrayOutputStream indexBlobStream = new ByteArrayOutputStream(); // no need to close ByteArrayOutputStream
return blobStoreManager.get(blobId, indexBlobStream, metadata, getDeleted).toCompletableFuture()
- .thenApplyAsync(f -> snapshotIndexSerde.fromBytes(indexBlobStream.toByteArray()), executor);
+ .thenApplyAsync(f -> {
+ try {
+ return snapshotIndexSerde.fromBytes(indexBlobStream.toByteArray());
+ } catch (Exception e) {
+ throw new SamzaException(String.format("Unable to deserialize SnapshotIndex bytes for blob ID: %s", blobId), e);
Review Comment:
could we use `.handle` for the exception instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org