You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/06/03 02:33:33 UTC
[GitHub] [hadoop-ozone] ChenSammi opened a new pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
ChenSammi opened a new pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012
https://issues.apache.org/jira/browse/HDDS-3658
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] ChenSammi removed a comment on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
ChenSammi removed a comment on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-660995694
/retest
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-651596050
As there is no other objection / comment I will merge this after a green build.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-638612932
> The rat check failed in doc module. Have we changed the rule recently?
Thanks @ChenSammi for flagging this problem. Posted #1017 to fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-660995694
/retest
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] github-actions[bot] commented on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-662971612
To re-run CI checks, please follow these steps with the source branch checked out:
```
git commit --allow-empty -m 'trigger new CI check'
git push
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#discussion_r453186007
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyInfo.java
##########
@@ -408,6 +421,15 @@ public KeyInfo getProtobuf() {
if (encInfo != null) {
kb.setFileEncryptionInfo(OMPBHelper.convert(encInfo));
}
+ if (ignorePipeline) {
+ kb.addAllKeyLocationList(keyLocationVersions.stream()
+ .map(OmKeyLocationInfoGroup::getCompactProtobuf)
+ .collect(Collectors.toList()));
+ } else {
+ kb.addAllKeyLocationList(keyLocationVersions.stream()
+ .map(OmKeyLocationInfoGroup::getProtobuf)
+ .collect(Collectors.toList()));
+ }
Review comment:
OmKeyInfo was changed in #1034 to avoid streams. `keyLocationVersions` is now converted in a loop:
https://github.com/apache/hadoop-ozone/blob/2af6198686d81daa3ad0513f723118637d2945cf/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyInfo.java#L388-L391
That's where normal/compact protobuf conversion should happen depending on `ignorePipeline` parameter.
##########
File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyLocationInfoGroup.java
##########
@@ -58,11 +58,22 @@ public long getVersion() {
return locationList;
}
+ public KeyLocationList getCompactProtobuf() {
+ return KeyLocationList.newBuilder()
+ .setVersion(version)
+ .addAllKeyLocations(
+ locationList.stream().map(
+ OmKeyLocationInfo::getCompactProtobuf)
+ .collect(Collectors.toList()))
+ .build();
+ }
+
Review comment:
Can we avoid duplication by extracting to a helper method that takes `OmKeyLocationInfo` conversion function (`OmKeyLocationInfo::getCompactProtobuf` or `OmKeyLocationInfo::getProtobuf`)?
##########
File path: Dockerfile
##########
@@ -0,0 +1,20 @@
+ARG BASE=latest
+FROM flokkr/base:35
Review comment:
This dockerfile seems to be unrelated to the change in this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-660989957
@elek and @adoroszlai , thanks for the review and feedback. I have rebased and updated the patch accordingly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] github-actions[bot] removed a comment on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
github-actions[bot] removed a comment on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-660996054
To re-run CI checks, please follow these steps with the source branch checked out:
```
git commit --allow-empty -m 'trigger new CI check'
git push
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] ChenSammi removed a comment on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
ChenSammi removed a comment on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-662971135
/retest
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-662971135
/retest
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] ChenSammi merged pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
ChenSammi merged pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] github-actions[bot] commented on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-660996054
To re-run CI checks, please follow these steps with the source branch checked out:
```
git commit --allow-empty -m 'trigger new CI check'
git push
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-638581173
The rat check failed in doc module. Have we changed the rule recently?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-663318814
Thanks @elek and @adoroszlai for review the code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] ChenSammi edited a comment on pull request #1012: HDDS-3658. Stop to persist container related pipeline info of each ke…
Posted by GitBox <gi...@apache.org>.
ChenSammi edited a comment on pull request #1012:
URL: https://github.com/apache/hadoop-ozone/pull/1012#issuecomment-638581173
The rat check failed in doc module. Have we changed the rule recently?
The failed UT is irrelavent.
testMultiBlockWrites3(org.apache.hadoop.ozone.client.rpc.TestCloseContainerHandlingByClient) Time elapsed: 300.019 s <<< ERROR!
java.lang.Exception: test timed out after 300000 milliseconds
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org