You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "GuoHao (Jira)" <ji...@apache.org> on 2023/11/20 08:47:00 UTC
[jira] [Comment Edited] (HDDS-8947) Intermittent failure in TestOzoneAtRestEncryption streaming
[ https://issues.apache.org/jira/browse/HDDS-8947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17787882#comment-17787882 ]
GuoHao edited comment on HDDS-8947 at 11/20/23 8:46 AM:
--------------------------------------------------------
I see [~adoroszlai] thanks for your report
was (Author: nicholas niu):
I see [~adoroszlai] thanks for you report
> Intermittent failure in TestOzoneAtRestEncryption streaming
> -----------------------------------------------------------
>
> Key: HDDS-8947
> URL: https://issues.apache.org/jira/browse/HDDS-8947
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Attila Doroszlai
> Assignee: GuoHao
> Priority: Major
>
> Intermittent failure in {{TestOzoneAtRestEncryption#testMPUwithOneStreamPart}}.
> {code}
> TestOzoneAtRestEncryption.testMPUwithOneStreamPart Time elapsed: 1.927 s <<< ERROR!
> org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException: Unable to find the block with bcsID 7 .Container 18 bcsId is 5.
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.validateContainerResponse(ContainerProtocolCalls.java:707)
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.lambda$createValidators$4(ContainerProtocolCalls.java:718)
> at org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommandWithRetry(XceiverClientGrpc.java:407)
> at org.apache.hadoop.hdds.scm.XceiverClientGrpc.lambda$sendCommandWithTraceIDAndRetry$0(XceiverClientGrpc.java:347)
> at org.apache.hadoop.hdds.tracing.TracingUtil.executeInSpan(TracingUtil.java:169)
> at org.apache.hadoop.hdds.tracing.TracingUtil.executeInNewSpan(TracingUtil.java:149)
> at org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommandWithTraceIDAndRetry(XceiverClientGrpc.java:342)
> at org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommand(XceiverClientGrpc.java:323)
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.getBlock(ContainerProtocolCalls.java:208)
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.lambda$getBlock$0(ContainerProtocolCalls.java:186)
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.tryEachDatanode(ContainerProtocolCalls.java:146)
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.getBlock(ContainerProtocolCalls.java:185)
> at org.apache.hadoop.hdds.scm.storage.BlockInputStream.getChunkInfos(BlockInputStream.java:255)
> at org.apache.hadoop.hdds.scm.storage.BlockInputStream.initialize(BlockInputStream.java:146)
> at org.apache.hadoop.hdds.scm.storage.BlockInputStream.readWithStrategy(BlockInputStream.java:308)
> at org.apache.hadoop.hdds.scm.storage.ExtendedInputStream.read(ExtendedInputStream.java:66)
> at org.apache.hadoop.hdds.scm.storage.ByteBufferReader.readFromBlock(ByteBufferReader.java:56)
> at org.apache.hadoop.hdds.scm.storage.MultipartInputStream.readWithStrategy(MultipartInputStream.java:96)
> at org.apache.hadoop.hdds.scm.storage.ExtendedInputStream.read(ExtendedInputStream.java:66)
> at org.apache.hadoop.crypto.CryptoInputStream.read(CryptoInputStream.java:203)
> at org.apache.hadoop.ozone.client.io.OzoneCryptoInputStream.read(OzoneCryptoInputStream.java:130)
> at org.apache.hadoop.hdds.scm.storage.ByteArrayReader.readFromBlock(ByteArrayReader.java:57)
> at org.apache.hadoop.hdds.scm.storage.MultipartInputStream.readWithStrategy(MultipartInputStream.java:96)
> at org.apache.hadoop.hdds.scm.storage.ExtendedInputStream.read(ExtendedInputStream.java:56)
> at org.apache.hadoop.ozone.client.io.OzoneInputStream.read(OzoneInputStream.java:56)
> at org.apache.hadoop.ozone.client.rpc.TestOzoneAtRestEncryption.testMultipartUploadWithEncryption(TestOzoneAtRestEncryption.java:567)
> at org.apache.hadoop.ozone.client.rpc.TestOzoneAtRestEncryption.testMPUwithOneStreamPart(TestOzoneAtRestEncryption.java:474)
> {code}
> and
> {code}
> TestOzoneAtRestEncryption.testMPUwithOneStreamPart Time elapsed: 1.263 s <<< ERROR!
> org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException: bcsId 22 mismatches with existing block Id 20 for block conID: 9 locID: 111677748019200024 bcsId: 22.
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.validateContainerResponse(ContainerProtocolCalls.java:707)
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.lambda$createValidators$4(ContainerProtocolCalls.java:718)
> at org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommandWithRetry(XceiverClientGrpc.java:407)
> at org.apache.hadoop.hdds.scm.XceiverClientGrpc.lambda$sendCommandWithTraceIDAndRetry$0(XceiverClientGrpc.java:347)
> at org.apache.hadoop.hdds.tracing.TracingUtil.executeInSpan(TracingUtil.java:169)
> at org.apache.hadoop.hdds.tracing.TracingUtil.executeInNewSpan(TracingUtil.java:149)
> at org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommandWithTraceIDAndRetry(XceiverClientGrpc.java:342)
> at org.apache.hadoop.hdds.scm.XceiverClientGrpc.sendCommand(XceiverClientGrpc.java:323)
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.getBlock(ContainerProtocolCalls.java:208)
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.lambda$getBlock$0(ContainerProtocolCalls.java:186)
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.tryEachDatanode(ContainerProtocolCalls.java:146)
> at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.getBlock(ContainerProtocolCalls.java:185)
> at org.apache.hadoop.hdds.scm.storage.BlockInputStream.getChunkInfos(BlockInputStream.java:255)
> at org.apache.hadoop.hdds.scm.storage.BlockInputStream.initialize(BlockInputStream.java:146)
> at org.apache.hadoop.hdds.scm.storage.BlockInputStream.readWithStrategy(BlockInputStream.java:308)
> at org.apache.hadoop.hdds.scm.storage.ExtendedInputStream.read(ExtendedInputStream.java:66)
> at org.apache.hadoop.hdds.scm.storage.ByteBufferReader.readFromBlock(ByteBufferReader.java:56)
> at org.apache.hadoop.hdds.scm.storage.MultipartInputStream.readWithStrategy(MultipartInputStream.java:96)
> at org.apache.hadoop.hdds.scm.storage.ExtendedInputStream.read(ExtendedInputStream.java:66)
> at org.apache.hadoop.crypto.CryptoInputStream.read(CryptoInputStream.java:189)
> at org.apache.hadoop.ozone.client.io.OzoneCryptoInputStream.read(OzoneCryptoInputStream.java:130)
> at org.apache.hadoop.hdds.scm.storage.ByteArrayReader.readFromBlock(ByteArrayReader.java:57)
> at org.apache.hadoop.hdds.scm.storage.MultipartInputStream.readWithStrategy(MultipartInputStream.java:96)
> at org.apache.hadoop.hdds.scm.storage.ExtendedInputStream.read(ExtendedInputStream.java:56)
> at org.apache.hadoop.ozone.client.io.OzoneInputStream.read(OzoneInputStream.java:56)
> at org.apache.hadoop.ozone.client.rpc.TestOzoneAtRestEncryption.testMultipartUploadWithEncryption(TestOzoneAtRestEncryption.java:567)
> at org.apache.hadoop.ozone.client.rpc.TestOzoneAtRestEncryption.testMPUwithOneStreamPart(TestOzoneAtRestEncryption.java:474)
> {code}
> * https://github.com/adoroszlai/ozone-build-results/blob/master/2023/06/23/23670/it-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneAtRestEncryption.txt
> * https://github.com/adoroszlai/ozone-build-results/blob/master/2023/06/23/23700/it-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneAtRestEncryption.txt
> * https://github.com/adoroszlai/ozone-build-results/blob/master/2023/06/28/23827/it-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneAtRestEncryption.txt
> * https://github.com/adoroszlai/ozone-build-results/blob/master/2023/07/13/24201/it-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneAtRestEncryption.txt
> Also in {{testMPUwithTwoStreamParts}}
> * https://github.com/adoroszlai/ozone-build-results/blob/master/2023/06/30/23960/it-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneAtRestEncryption.txt
> * https://github.com/adoroszlai/ozone-build-results/blob/master/2023/07/13/24212/it-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneAtRestEncryption.txt
> * https://github.com/adoroszlai/ozone-build-results/blob/master/2023/07/16/24280/it-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneAtRestEncryption.txt
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org