You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/12/13 07:45:09 UTC
[GitHub] [ozone] Cyrill opened a new pull request, #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Cyrill opened a new pull request, #4073:
URL: https://github.com/apache/ozone/pull/4073
## What changes were proposed in this pull request?
The goal of this refactoring is to replace the code to read from a list of streams in BlockInputStream with MultipartInputStream.
- Instead of storing the list of streams and calculating offsets in-place we delegate to MultipartInputStream
- Now MultipartInputStream has three subclasses, changed the class to match them all
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7624
## How was this patch tested?
Existing unit tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] Cyrill commented on pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Posted by GitBox <gi...@apache.org>.
Cyrill commented on PR #4073:
URL: https://github.com/apache/ozone/pull/4073#issuecomment-1375574034
@kerneltime @umamaheswararao @jojochuang Could you please take a look at the changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-7624. Reuse MultipartInputStream in BlockInputStream [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #4073:
URL: https://github.com/apache/ozone/pull/4073#issuecomment-2518113921
/pending please rebase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4073:
URL: https://github.com/apache/ozone/pull/4073#discussion_r1097079308
##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockInputStream.java:
##########
@@ -279,139 +252,38 @@ protected ChunkInputStream createChunkInputStream(ChunkInfo chunkInfo) {
protected synchronized int readWithStrategy(ByteReaderStrategy strategy)
throws IOException {
Preconditions.checkArgument(strategy != null);
- if (!initialized) {
+ if (!isInitialized()) {
initialize();
}
-
- checkOpen();
- int totalReadLen = 0;
- int len = strategy.getTargetLength();
- while (len > 0) {
- // if we are at the last chunk and have read the entire chunk, return
- if (chunkStreams.size() == 0 ||
- (chunkStreams.size() - 1 <= chunkIndex &&
- chunkStreams.get(chunkIndex)
- .getRemaining() == 0)) {
- return totalReadLen == 0 ? EOF : totalReadLen;
- }
-
- // Get the current chunkStream and read data from it
- ChunkInputStream current = chunkStreams.get(chunkIndex);
- int numBytesToRead = Math.min(len, (int)current.getRemaining());
- int numBytesRead;
- try {
- numBytesRead = strategy.readFromBlock(current, numBytesToRead);
- retries = 0;
- // If we get a StorageContainerException or an IOException due to
- // datanodes are not reachable, refresh to get the latest pipeline
- // info and retry.
- // Otherwise, just retry according to the retry policy.
- } catch (SCMSecurityException ex) {
- throw ex;
- } catch (StorageContainerException e) {
- if (shouldRetryRead(e)) {
- handleReadError(e);
- continue;
- } else {
- throw e;
- }
- } catch (IOException ex) {
- if (shouldRetryRead(ex)) {
- if (isConnectivityIssue(ex)) {
- handleReadError(ex);
- } else {
- current.releaseClient();
- }
- continue;
- } else {
- throw ex;
- }
- }
-
- if (numBytesRead != numBytesToRead) {
- // This implies that there is either data loss or corruption in the
- // chunk entries. Even EOF in the current stream would be covered in
- // this case.
- throw new IOException(String.format(
- "Inconsistent read for chunkName=%s length=%d numBytesToRead= %d " +
- "numBytesRead=%d", current.getChunkName(), current.getLength(),
- numBytesToRead, numBytesRead));
- }
- totalReadLen += numBytesRead;
- len -= numBytesRead;
- if (current.getRemaining() <= 0 &&
- ((chunkIndex + 1) < chunkStreams.size())) {
- chunkIndex += 1;
- }
- }
- return totalReadLen;
+ return delegateStream.readWithStrategy(strategy);
}
/**
* Seeks the BlockInputStream to the specified position. If the stream is
* not initialized, save the seeked position via blockPosition. Otherwise,
* update the position in 2 steps:
- * 1. Updating the chunkIndex to the chunkStream corresponding to the
- * seeked position.
- * 2. Seek the corresponding chunkStream to the adjusted position.
- *
+ * 1. Updating the chunkIndex to the chunkStream corresponding to the
+ * seeked position.
+ * 2. Seek the corresponding chunkStream to the adjusted position.
+ * <p>
Review Comment:
Maybe this "< p >" was added by accident?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4073:
URL: https://github.com/apache/ozone/pull/4073#discussion_r1097079308
##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockInputStream.java:
##########
@@ -279,139 +252,38 @@ protected ChunkInputStream createChunkInputStream(ChunkInfo chunkInfo) {
protected synchronized int readWithStrategy(ByteReaderStrategy strategy)
throws IOException {
Preconditions.checkArgument(strategy != null);
- if (!initialized) {
+ if (!isInitialized()) {
initialize();
}
-
- checkOpen();
- int totalReadLen = 0;
- int len = strategy.getTargetLength();
- while (len > 0) {
- // if we are at the last chunk and have read the entire chunk, return
- if (chunkStreams.size() == 0 ||
- (chunkStreams.size() - 1 <= chunkIndex &&
- chunkStreams.get(chunkIndex)
- .getRemaining() == 0)) {
- return totalReadLen == 0 ? EOF : totalReadLen;
- }
-
- // Get the current chunkStream and read data from it
- ChunkInputStream current = chunkStreams.get(chunkIndex);
- int numBytesToRead = Math.min(len, (int)current.getRemaining());
- int numBytesRead;
- try {
- numBytesRead = strategy.readFromBlock(current, numBytesToRead);
- retries = 0;
- // If we get a StorageContainerException or an IOException due to
- // datanodes are not reachable, refresh to get the latest pipeline
- // info and retry.
- // Otherwise, just retry according to the retry policy.
- } catch (SCMSecurityException ex) {
- throw ex;
- } catch (StorageContainerException e) {
- if (shouldRetryRead(e)) {
- handleReadError(e);
- continue;
- } else {
- throw e;
- }
- } catch (IOException ex) {
- if (shouldRetryRead(ex)) {
- if (isConnectivityIssue(ex)) {
- handleReadError(ex);
- } else {
- current.releaseClient();
- }
- continue;
- } else {
- throw ex;
- }
- }
-
- if (numBytesRead != numBytesToRead) {
- // This implies that there is either data loss or corruption in the
- // chunk entries. Even EOF in the current stream would be covered in
- // this case.
- throw new IOException(String.format(
- "Inconsistent read for chunkName=%s length=%d numBytesToRead= %d " +
- "numBytesRead=%d", current.getChunkName(), current.getLength(),
- numBytesToRead, numBytesRead));
- }
- totalReadLen += numBytesRead;
- len -= numBytesRead;
- if (current.getRemaining() <= 0 &&
- ((chunkIndex + 1) < chunkStreams.size())) {
- chunkIndex += 1;
- }
- }
- return totalReadLen;
+ return delegateStream.readWithStrategy(strategy);
}
/**
* Seeks the BlockInputStream to the specified position. If the stream is
* not initialized, save the seeked position via blockPosition. Otherwise,
* update the position in 2 steps:
- * 1. Updating the chunkIndex to the chunkStream corresponding to the
- * seeked position.
- * 2. Seek the corresponding chunkStream to the adjusted position.
- *
+ * 1. Updating the chunkIndex to the chunkStream corresponding to the
+ * seeked position.
+ * 2. Seek the corresponding chunkStream to the adjusted position.
+ * <p>
Review Comment:
Maybe this "<p>" was added by accident?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-7624. Reuse MultipartInputStream in BlockInputStream [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #4073:
URL: https://github.com/apache/ozone/pull/4073#issuecomment-2563188768
Thank you very much for the patch. I am closing this PR __temporarily__ as there was no activity recently and it is waiting for response from its author.
It doesn't mean that this PR is not important or ignored: feel free to reopen the PR at any time.
It only means that attention of committers is not required. We prefer to keep the review queue clean. This ensures PRs in need of review are more visible, which results in faster feedback for all PRs.
If you need ANY help to finish this PR, please [contact the community](https://github.com/apache/hadoop-ozone#contact) on the mailing list or the slack channel."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kerneltime commented on pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Posted by GitBox <gi...@apache.org>.
kerneltime commented on PR #4073:
URL: https://github.com/apache/ozone/pull/4073#issuecomment-1357994339
@Cyrill thank you for the change. I am curious if this is for code reuse/code cleanup or if there are any performance of functional differences desired from this code change.
cc @swamirishi @aswinshakil can you'll please take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] Cyrill commented on pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Posted by GitBox <gi...@apache.org>.
Cyrill commented on PR #4073:
URL: https://github.com/apache/ozone/pull/4073#issuecomment-1359230821
@kerneltime The goal was to reuse the code and make it easier to understand.
I noticed the same code pattern with subtle differences while debugging the ozone client and decided to separate the stream calculation from RPCs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] Cyrill commented on a diff in pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Posted by GitBox <gi...@apache.org>.
Cyrill commented on code in PR #4073:
URL: https://github.com/apache/ozone/pull/4073#discussion_r1067154323
##########
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/MultipartCryptoKeyInputStream.java:
##########
@@ -0,0 +1,46 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.ozone.client.io;
+
+import org.apache.hadoop.hdds.scm.storage.ByteReaderStrategy;
+import org.apache.hadoop.hdds.scm.storage.MultipartInputStream;
+import org.apache.hadoop.hdds.scm.storage.PartInputStream;
+
+import java.util.List;
+
+/**
+ * {@link MultipartInputStream} for accessing MPU keys in encrypted buckets.
+ * see also {@link OzoneCryptoInputStream}
+ */
+public class MultipartCryptoKeyInputStream extends MultipartInputStream {
+
+ public MultipartCryptoKeyInputStream(
+ String keyName,
+ List<? extends PartInputStream> inputStreams) {
+ super(" Key: " + keyName, inputStreams);
Review Comment:
It's used as a prefix in logging and exceptions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] Cyrill commented on a diff in pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Posted by GitBox <gi...@apache.org>.
Cyrill commented on code in PR #4073:
URL: https://github.com/apache/ozone/pull/4073#discussion_r1067184964
##########
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/MultipartCryptoKeyInputStream.java:
##########
@@ -0,0 +1,46 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.ozone.client.io;
Review Comment:
It's used in the client code only, thus added to the client module. The same place the original file was added to (before the refactoring)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] Cyrill commented on pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Posted by GitBox <gi...@apache.org>.
Cyrill commented on PR #4073:
URL: https://github.com/apache/ozone/pull/4073#issuecomment-1363864676
@kerneltime @umamaheswararao @jojochuang Could you please take a look at the changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4073:
URL: https://github.com/apache/ozone/pull/4073#discussion_r1097079308
##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockInputStream.java:
##########
@@ -279,139 +252,38 @@ protected ChunkInputStream createChunkInputStream(ChunkInfo chunkInfo) {
protected synchronized int readWithStrategy(ByteReaderStrategy strategy)
throws IOException {
Preconditions.checkArgument(strategy != null);
- if (!initialized) {
+ if (!isInitialized()) {
initialize();
}
-
- checkOpen();
- int totalReadLen = 0;
- int len = strategy.getTargetLength();
- while (len > 0) {
- // if we are at the last chunk and have read the entire chunk, return
- if (chunkStreams.size() == 0 ||
- (chunkStreams.size() - 1 <= chunkIndex &&
- chunkStreams.get(chunkIndex)
- .getRemaining() == 0)) {
- return totalReadLen == 0 ? EOF : totalReadLen;
- }
-
- // Get the current chunkStream and read data from it
- ChunkInputStream current = chunkStreams.get(chunkIndex);
- int numBytesToRead = Math.min(len, (int)current.getRemaining());
- int numBytesRead;
- try {
- numBytesRead = strategy.readFromBlock(current, numBytesToRead);
- retries = 0;
- // If we get a StorageContainerException or an IOException due to
- // datanodes are not reachable, refresh to get the latest pipeline
- // info and retry.
- // Otherwise, just retry according to the retry policy.
- } catch (SCMSecurityException ex) {
- throw ex;
- } catch (StorageContainerException e) {
- if (shouldRetryRead(e)) {
- handleReadError(e);
- continue;
- } else {
- throw e;
- }
- } catch (IOException ex) {
- if (shouldRetryRead(ex)) {
- if (isConnectivityIssue(ex)) {
- handleReadError(ex);
- } else {
- current.releaseClient();
- }
- continue;
- } else {
- throw ex;
- }
- }
-
- if (numBytesRead != numBytesToRead) {
- // This implies that there is either data loss or corruption in the
- // chunk entries. Even EOF in the current stream would be covered in
- // this case.
- throw new IOException(String.format(
- "Inconsistent read for chunkName=%s length=%d numBytesToRead= %d " +
- "numBytesRead=%d", current.getChunkName(), current.getLength(),
- numBytesToRead, numBytesRead));
- }
- totalReadLen += numBytesRead;
- len -= numBytesRead;
- if (current.getRemaining() <= 0 &&
- ((chunkIndex + 1) < chunkStreams.size())) {
- chunkIndex += 1;
- }
- }
- return totalReadLen;
+ return delegateStream.readWithStrategy(strategy);
}
/**
* Seeks the BlockInputStream to the specified position. If the stream is
* not initialized, save the seeked position via blockPosition. Otherwise,
* update the position in 2 steps:
- * 1. Updating the chunkIndex to the chunkStream corresponding to the
- * seeked position.
- * 2. Seek the corresponding chunkStream to the adjusted position.
- *
+ * 1. Updating the chunkIndex to the chunkStream corresponding to the
+ * seeked position.
+ * 2. Seek the corresponding chunkStream to the adjusted position.
+ * <p>
Review Comment:
Maybe this <p> was added by accident?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] Cyrill commented on pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Posted by GitBox <gi...@apache.org>.
Cyrill commented on PR #4073:
URL: https://github.com/apache/ozone/pull/4073#issuecomment-1350775841
@adoroszlai Please review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-7624. Reuse MultipartInputStream in BlockInputStream [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
URL: https://github.com/apache/ozone/pull/4073
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-7624. Reuse MultipartInputStream in BlockInputStream [ozone]
Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime commented on PR #4073:
URL: https://github.com/apache/ozone/pull/4073#issuecomment-2166164223
@Cyrill do you plan to continue with this PR? Can you please rebase?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] jojochuang commented on a diff in pull request #4073: HDDS-7624. Reuse MultipartInputStream in BlockInputStream
Posted by GitBox <gi...@apache.org>.
jojochuang commented on code in PR #4073:
URL: https://github.com/apache/ozone/pull/4073#discussion_r1065190288
##########
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/MultipartCryptoKeyInputStream.java:
##########
@@ -0,0 +1,46 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.ozone.client.io;
Review Comment:
would it make sense to move over to org.apache.hadoop.hdds.scm.storage package? Similar to MultipartInputStream.
##########
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/MultipartCryptoKeyInputStream.java:
##########
@@ -0,0 +1,46 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.ozone.client.io;
+
+import org.apache.hadoop.hdds.scm.storage.ByteReaderStrategy;
+import org.apache.hadoop.hdds.scm.storage.MultipartInputStream;
+import org.apache.hadoop.hdds.scm.storage.PartInputStream;
+
+import java.util.List;
+
+/**
+ * {@link MultipartInputStream} for accessing MPU keys in encrypted buckets.
+ * see also {@link OzoneCryptoInputStream}
+ */
+public class MultipartCryptoKeyInputStream extends MultipartInputStream {
+
+ public MultipartCryptoKeyInputStream(
+ String keyName,
+ List<? extends PartInputStream> inputStreams) {
+ super(" Key: " + keyName, inputStreams);
Review Comment:
why use this (" Key: " + keyName) as the key name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org