You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/07/01 03:04:12 UTC
[GitHub] [hadoop-ozone] GlenGeng opened a new pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl
GlenGeng opened a new pull request #1151:
URL: https://github.com/apache/hadoop-ozone/pull/1151
## What changes were proposed in this pull request?
switch from SCMPipelineManager to PipelineManagerV2Impl.
mainly focus on fixing unit test
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3191
## How was this patch tested?
unit tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] xiaoyuyao merged pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl
Posted by GitBox <gi...@apache.org>.
xiaoyuyao merged pull request #1151:
URL: https://github.com/apache/hadoop-ozone/pull/1151
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] GlenGeng commented on pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl
Posted by GitBox <gi...@apache.org>.
GlenGeng commented on pull request #1151:
URL: https://github.com/apache/hadoop-ozone/pull/1151#issuecomment-652947720
Hello @elek @adoroszlai Would you please help check a GitHub CI problem ?
I want to push 2 commit from my dev branch `GlenGeng:HDDS-3191` to branch `apache:HDDS-2823`, meanwhile I also tried CI in my personal repo (submit a PR from branch `GlenGeng:HDDS-3191` to branch `GlenGeng:HDDS-2823`, see https://github.com/GlenGeng/hadoop-ozone/pull/2).
It is pretty sure that branch `apache:HDDS-2823` is the same as branch `GlenGeng:HDDS-2823`. I have succeed in my [person repo](https://github.com/GlenGeng/hadoop-ozone/pull/2/checks), but failed several time in [apache repo](https://github.com/apache/hadoop-ozone/pull/1151/checks?check_run_id=829732353).
Seems these failed cases are related to raft leader election, which has randomness. Would you please help have a look ?
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] timmylicheng commented on pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl
Posted by GitBox <gi...@apache.org>.
timmylicheng commented on pull request #1151:
URL: https://github.com/apache/hadoop-ozone/pull/1151#issuecomment-655479330
LGTM, +1.
See if failed tests can be cleared.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] GlenGeng commented on a change in pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl
Posted by GitBox <gi...@apache.org>.
GlenGeng commented on a change in pull request #1151:
URL: https://github.com/apache/hadoop-ozone/pull/1151#discussion_r451254135
##########
File path: hadoop-ozone/recon/pom.xml
##########
@@ -108,6 +108,7 @@
</goals>
<configuration>
<arguments>pnpm config set store-dir ~/.pnpm-store</arguments>
+ <npmInheritsProxyConfigFromMaven>false</npmInheritsProxyConfigFromMaven>
Review comment:
Thanks for pointing out! Will drop this change.
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMHAConfiguration.java
##########
@@ -78,7 +78,7 @@
description = "The size of the raft segment used by Apache Ratis on" +
" SCM. (16 KB by default)"
)
- private long raftSegmentSize = 16L * 1024L;
+ private double raftSegmentSize = 16L * 1024L;
Review comment:
you will see that `ConfigType.SIZE` will be reflected as `StorageUnit`, which need be `double`.
refer to `ConfigurationReflectionUtil`,
```
case SIZE:
forcedFieldSet(field, configuration,
from.getStorageSize(key, "0B", configAnnotation.sizeUnit()));
break;
```
and `ConfigurationSource`
```
default double getStorageSize(String name, String defaultValue,
StorageUnit targetUnit)
```
This code is not reached before, so merged into HDDS-2823 without breaking CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl
Posted by GitBox <gi...@apache.org>.
xiaoyuyao commented on a change in pull request #1151:
URL: https://github.com/apache/hadoop-ozone/pull/1151#discussion_r454525325
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMHAConfiguration.java
##########
@@ -78,7 +78,7 @@
description = "The size of the raft segment used by Apache Ratis on" +
" SCM. (16 KB by default)"
)
- private long raftSegmentSize = 16L * 1024L;
+ private double raftSegmentSize = 16L * 1024L;
Review comment:
I see. Thanks for the details.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl
Posted by GitBox <gi...@apache.org>.
xiaoyuyao commented on a change in pull request #1151:
URL: https://github.com/apache/hadoop-ozone/pull/1151#discussion_r451155274
##########
File path: hadoop-ozone/recon/pom.xml
##########
@@ -108,6 +108,7 @@
</goals>
<configuration>
<arguments>pnpm config set store-dir ~/.pnpm-store</arguments>
+ <npmInheritsProxyConfigFromMaven>false</npmInheritsProxyConfigFromMaven>
Review comment:
This change seems unrelated. Can you merge it from master?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] GlenGeng commented on pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl
Posted by GitBox <gi...@apache.org>.
GlenGeng commented on pull request #1151:
URL: https://github.com/apache/hadoop-ozone/pull/1151#issuecomment-653330301
also see this flaky case failure in `master` branch
see https://github.com/apache/hadoop-ozone/pull/1147
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl
Posted by GitBox <gi...@apache.org>.
xiaoyuyao commented on a change in pull request #1151:
URL: https://github.com/apache/hadoop-ozone/pull/1151#discussion_r451149178
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMHAConfiguration.java
##########
@@ -78,7 +78,7 @@
description = "The size of the raft segment used by Apache Ratis on" +
" SCM. (16 KB by default)"
)
- private long raftSegmentSize = 16L * 1024L;
+ private double raftSegmentSize = 16L * 1024L;
Review comment:
Is there a reason to change long to double here and force conversion later?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] xiaoyuyao commented on pull request #1151: HDDS-3191: switch from SCMPipelineManager to PipelineManagerV2Impl
Posted by GitBox <gi...@apache.org>.
xiaoyuyao commented on pull request #1151:
URL: https://github.com/apache/hadoop-ozone/pull/1151#issuecomment-658313263
+1, I will merge it shortly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org