You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/09/01 00:38:04 UTC

[GitHub] [hadoop-ozone] maobaolong opened a new pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

maobaolong opened a new pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369


   
   ## What changes were proposed in this pull request?
   
   Provide a way to get the default value and key of java-based-configuration easily.
   
   ## What is the link to the Apache JIRA
   
   HDDS-4104
   
   ## How was this patch tested?
   
   This PR with the related unit tests.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong closed pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong closed pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong commented on a change in pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong commented on a change in pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#discussion_r482188898



##########
File path: hadoop-hdds/config/src/test/java/org/apache/hadoop/hdds/conf/TestConfigurationReflectionUtil.java
##########
@@ -0,0 +1,71 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.conf;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.util.Optional;
+
+/**
+ * Test the configuration reflection utility class.
+ */
+public class TestConfigurationReflectionUtil {

Review comment:
       Ok, that's good cases, I have add them.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong commented on pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong commented on pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#issuecomment-692396408


   @elek Please take a look at this PR, thanks.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai merged pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
adoroszlai merged pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong commented on pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong commented on pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#issuecomment-686590851


   @adoroszlai Thanks for your help during the reviewing.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong commented on pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong commented on pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#issuecomment-685847325


   @adoroszlai Thanks for your review and the good suggestion, I push new commits and addressed the comments inline.
   PTAL.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#discussion_r482920233



##########
File path: hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/ConfigurationReflectionUtil.java
##########
@@ -240,4 +242,49 @@ private static ConfigType detectConfigType(Class<?> parameterType,
       }
     }
   }
+
+  public static Optional<String> getDefaultValue(Class<?> configClass,
+      String fieldName) {
+    return findFieldConfigAnnotationByName(configClass, fieldName)
+        .map(Config::defaultValue);
+  }
+
+  public static Optional<String> getKey(Class<?> configClass,
+      String fieldName) {
+    ConfigGroup configGroup =
+        configClass.getAnnotation(ConfigGroup.class);
+
+    return findFieldConfigAnnotationByName(configClass,
+        fieldName).map(
+            config -> configGroup == null ? config.key()
+                : configGroup.prefix() + "." + config.key());
+  }
+
+  public static Optional<ConfigType> getType(Class<?> configClass,
+      String fieldName) {
+    return findFieldConfigAnnotationByName(configClass, fieldName)
+        .map(Config::type);
+  }
+
+  private static Optional<Config> findFieldConfigAnnotationByName(
+      final Class<?> configClass, String fieldName) {
+    Class<?> theClass = configClass;
+    while (theClass != null) {
+      Optional<Config> config = Stream.of(theClass.getDeclaredFields())
+          .filter(f -> f.getName().equals(fieldName))
+          .findFirst()
+          .map(f -> f.getAnnotation(Config.class));
+
+      if (config.isPresent()) {
+        return config;
+      }
+
+      if (!theClass.getSuperclass().equals(Object.class)) {
+        theClass = theClass.getSuperclass();
+      } else {
+        theClass = null;
+      }

Review comment:
       The following change makes this method safe to invoke with `Object.class` directly (current code produces NPE).  It may be unlikely that someone does this, so I think it's OK to commit as is.
   
   ```suggestion
         theClass = theClass.getSuperclass();
         if (Object.class.equals(theClass)) {
           theClass = null;
         }
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong closed pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong closed pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#issuecomment-694112238


   Thanks @maobaolong for the contribution.  Merged to master.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong commented on pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong commented on pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#issuecomment-686098874


   @adoroszlai PTAL.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#discussion_r482225821



##########
File path: hadoop-ozone/dev-support/intellij/ozone-site.xml
##########
@@ -59,12 +59,12 @@
     <name>hdds.prometheus.endpoint.enabled</name>
     <value>true</value>
   </property>
-  <property>
-    <name>ozone.recon.address</name>
-    <value>localhost:9891</value>
-  </property>
-  <property>
-    <name>ozone.recon.db.dir</name>
-    <value>/tmp/recon</value>
-  </property>
+<!--  <property>-->
+<!--    <name>ozone.recon.address</name>-->
+<!--    <value>localhost:9891</value>-->
+<!--  </property>-->
+<!--  <property>-->
+<!--    <name>ozone.recon.db.dir</name>-->
+<!--    <value>/tmp/recon</value>-->
+<!--  </property>-->

Review comment:
       I think this is unintended change in this PR.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] codecov-commenter commented on pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#issuecomment-685897016


   # [Codecov](https://codecov.io/gh/apache/hadoop-ozone/pull/1369?src=pr&el=h1) Report
   > Merging [#1369](https://codecov.io/gh/apache/hadoop-ozone/pull/1369?src=pr&el=desc) into [master](https://codecov.io/gh/apache/hadoop-ozone/commit/d34ab29b622168c57bd2d3049d060c42735de7f4?el=desc) will **decrease** coverage by `0.19%`.
   > The diff coverage is `100.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/graphs/tree.svg?width=650&height=150&src=pr&token=5YeeptJMby)](https://codecov.io/gh/apache/hadoop-ozone/pull/1369?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1369      +/-   ##
   ============================================
   - Coverage     75.32%   75.13%   -0.20%     
   + Complexity    10502    10501       -1     
   ============================================
     Files           990      990              
     Lines         50757    50867     +110     
     Branches       4951     4959       +8     
   ============================================
   - Hits          38233    38219      -14     
   - Misses        10158    10258     +100     
   - Partials       2366     2390      +24     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/hadoop-ozone/pull/1369?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [.../hadoop/hdds/conf/ConfigurationReflectionUtil.java](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/diff?src=pr&el=tree#diff-aGFkb29wLWhkZHMvY29uZmlnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3AvaGRkcy9jb25mL0NvbmZpZ3VyYXRpb25SZWZsZWN0aW9uVXRpbC5qYXZh) | `80.62% <100.00%> (+4.20%)` | `40.00 <11.00> (+11.00)` | |
   | [...hdds/scm/container/CloseContainerEventHandler.java](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/diff?src=pr&el=tree#diff-aGFkb29wLWhkZHMvc2VydmVyLXNjbS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaGFkb29wL2hkZHMvc2NtL2NvbnRhaW5lci9DbG9zZUNvbnRhaW5lckV2ZW50SGFuZGxlci5qYXZh) | `72.41% <0.00%> (-17.25%)` | `6.00% <0.00%> (ø%)` | |
   | [.../hadoop/ozone/recon/scm/ReconContainerManager.java](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/diff?src=pr&el=tree#diff-aGFkb29wLW96b25lL3JlY29uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3Avb3pvbmUvcmVjb24vc2NtL1JlY29uQ29udGFpbmVyTWFuYWdlci5qYXZh) | `83.05% <0.00%> (-13.56%)` | `13.00% <0.00%> (-1.00%)` | |
   | [...ache/hadoop/ozone/om/codec/S3SecretValueCodec.java](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/diff?src=pr&el=tree#diff-aGFkb29wLW96b25lL296b25lLW1hbmFnZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2hhZG9vcC9vem9uZS9vbS9jb2RlYy9TM1NlY3JldFZhbHVlQ29kZWMuamF2YQ==) | `90.90% <0.00%> (-9.10%)` | `3.00% <0.00%> (-1.00%)` | |
   | [...on/scm/ReconIncrementalContainerReportHandler.java](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/diff?src=pr&el=tree#diff-aGFkb29wLW96b25lL3JlY29uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3Avb3pvbmUvcmVjb24vc2NtL1JlY29uSW5jcmVtZW50YWxDb250YWluZXJSZXBvcnRIYW5kbGVyLmphdmE=) | `60.00% <0.00%> (-7.50%)` | `4.00% <0.00%> (ø%)` | |
   | [...va/org/apache/hadoop/hdds/utils/db/RDBMetrics.java](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/diff?src=pr&el=tree#diff-aGFkb29wLWhkZHMvZnJhbWV3b3JrL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3AvaGRkcy91dGlscy9kYi9SREJNZXRyaWNzLmphdmE=) | `92.85% <0.00%> (-7.15%)` | `14.00% <0.00%> (-1.00%)` | |
   | [...doop/hdds/scm/container/ContainerStateManager.java](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/diff?src=pr&el=tree#diff-aGFkb29wLWhkZHMvc2VydmVyLXNjbS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaGFkb29wL2hkZHMvc2NtL2NvbnRhaW5lci9Db250YWluZXJTdGF0ZU1hbmFnZXIuamF2YQ==) | `81.67% <0.00%> (-6.88%)` | `32.00% <0.00%> (-3.00%)` | |
   | [...rg/apache/hadoop/hdds/scm/server/SCMCertStore.java](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/diff?src=pr&el=tree#diff-aGFkb29wLWhkZHMvc2VydmVyLXNjbS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaGFkb29wL2hkZHMvc2NtL3NlcnZlci9TQ01DZXJ0U3RvcmUuamF2YQ==) | `38.23% <0.00%> (-5.89%)` | `5.00% <0.00%> (-1.00%)` | |
   | [.../org/apache/hadoop/hdds/scm/XceiverClientGrpc.java](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/diff?src=pr&el=tree#diff-aGFkb29wLWhkZHMvY2xpZW50L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3AvaGRkcy9zY20vWGNlaXZlckNsaWVudEdycGMuamF2YQ==) | `68.12% <0.00%> (-5.82%)` | `38.00% <0.00%> (ø%)` | |
   | [...p/ozone/security/OzoneDelegationTokenSelector.java](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/diff?src=pr&el=tree#diff-aGFkb29wLW96b25lL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaGFkb29wL296b25lL3NlY3VyaXR5L096b25lRGVsZWdhdGlvblRva2VuU2VsZWN0b3IuamF2YQ==) | `72.22% <0.00%> (-5.56%)` | `7.00% <0.00%> (-1.00%)` | |
   | ... and [33 more](https://codecov.io/gh/apache/hadoop-ozone/pull/1369/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/hadoop-ozone/pull/1369?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/hadoop-ozone/pull/1369?src=pr&el=footer). Last update [d34ab29...67c37eb](https://codecov.io/gh/apache/hadoop-ozone/pull/1369?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong commented on a change in pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong commented on a change in pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#discussion_r482199368



##########
File path: hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/ConfigurationReflectionUtil.java
##########
@@ -240,4 +242,52 @@ private static ConfigType detectConfigType(Class<?> parameterType,
       }
     }
   }
+
+  public static Optional<String> getDefaultValue(Class<?> configClass,
+      String fieldName) {
+    Config annotation = findFieldConfigAnnotationByName(configClass,
+        fieldName);
+    if (annotation != null) {
+      return Optional.of(annotation.defaultValue());
+    }
+    return Optional.empty();
+  }
+
+  public static Optional<String> getKey(Class<?> configClass,
+      String fieldName) {
+    ConfigGroup configGroup =
+        configClass.getAnnotation(ConfigGroup.class);
+
+    Config annotation = findFieldConfigAnnotationByName(configClass,
+        fieldName);
+    if (annotation != null) {
+      String key = annotation.key();
+      if (configGroup != null) {
+        key = configGroup.prefix() + "." + annotation.key();
+      }
+      return Optional.of(key);
+    }
+    return Optional.empty();
+  }
+
+  public static Optional<ConfigType> getType(Class<?> configClass,
+      String fieldName) {
+    Config config = findFieldConfigAnnotationByName(configClass,
+        fieldName);
+    if (config != null) {
+      return Optional.of(config.type());
+    }
+    return Optional.empty();
+  }
+
+  private static Config findFieldConfigAnnotationByName(Class<?> configClass,
+      String fieldName) {
+    Optional<Field> field = Stream.of(configClass.getDeclaredFields())
+        .filter(f -> f.getName().equals(fieldName))
+        .findFirst();
+    if (field.isPresent()) {
+      return field.get().getAnnotation(Config.class);
+    }
+    return null;

Review comment:
       It looks more clear than before, thanks, done.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#discussion_r480984328



##########
File path: hadoop-hdds/config/src/test/java/org/apache/hadoop/hdds/conf/TestConfigurationReflectionUtil.java
##########
@@ -0,0 +1,71 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.conf;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.util.Optional;
+
+/**
+ * Test the configuration reflection utility class.
+ */
+public class TestConfigurationReflectionUtil {
+
+  @Test
+  public void testClassWithConfigGroup() {
+    Optional<ConfigType> actualType =
+        ConfigurationReflectionUtil.getType(
+            ConfigurationExample.class, "waitTime");
+    Assert.assertTrue(actualType.isPresent());
+    Assert.assertEquals(ConfigType.TIME, actualType.get());
+
+    Optional<String> actualKey =
+        ConfigurationReflectionUtil.getKey(
+        ConfigurationExample.class, "waitTime");
+    Assert.assertTrue(actualKey.isPresent());
+    Assert.assertEquals("ozone.scm.client.wait", actualKey.get());
+
+    Optional<String> actualDefaultValue =
+        ConfigurationReflectionUtil.getDefaultValue(
+            ConfigurationExample.class, "waitTime");
+    Assert.assertTrue(actualKey.isPresent());
+    Assert.assertEquals("30m", actualDefaultValue.get());
+  }
+
+  @Test
+  public void testClassWithoutConfigGroup() {
+    Optional<ConfigType> actualType =
+        ConfigurationReflectionUtil.getType(
+            ConfigurationExampleGrandParent.class, "number");
+    Assert.assertTrue(actualType.isPresent());
+    Assert.assertEquals(ConfigType.AUTO, actualType.get());
+
+    Optional<String> actualKey =
+        ConfigurationReflectionUtil.getKey(
+            ConfigurationExampleGrandParent.class, "number");
+    Assert.assertTrue(actualKey.isPresent());
+    Assert.assertEquals("number", actualKey.get());
+
+    Optional<String> actualDefaultValue =
+        ConfigurationReflectionUtil.getDefaultValue(
+            ConfigurationExampleGrandParent.class, "number");
+    Assert.assertTrue(actualKey.isPresent());

Review comment:
       ```suggestion
       Assert.assertTrue(actualDefaultValue.isPresent());
   ```

##########
File path: hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/ConfigurationReflectionUtil.java
##########
@@ -240,4 +242,52 @@ private static ConfigType detectConfigType(Class<?> parameterType,
       }
     }
   }
+
+  public static Optional<String> getDefaultValue(Class<?> configClass,
+      String fieldName) {
+    Config annotation = findFieldConfigAnnotationByName(configClass,
+        fieldName);
+    if (annotation != null) {
+      return Optional.of(annotation.defaultValue());
+    }
+    return Optional.empty();
+  }
+
+  public static Optional<String> getKey(Class<?> configClass,
+      String fieldName) {
+    ConfigGroup configGroup =
+        configClass.getAnnotation(ConfigGroup.class);
+
+    Config annotation = findFieldConfigAnnotationByName(configClass,
+        fieldName);
+    if (annotation != null) {
+      String key = annotation.key();
+      if (configGroup != null) {
+        key = configGroup.prefix() + "." + annotation.key();
+      }
+      return Optional.of(key);
+    }
+    return Optional.empty();
+  }
+
+  public static Optional<ConfigType> getType(Class<?> configClass,
+      String fieldName) {
+    Config config = findFieldConfigAnnotationByName(configClass,
+        fieldName);
+    if (config != null) {
+      return Optional.of(config.type());
+    }
+    return Optional.empty();
+  }
+
+  private static Config findFieldConfigAnnotationByName(Class<?> configClass,
+      String fieldName) {
+    Optional<Field> field = Stream.of(configClass.getDeclaredFields())
+        .filter(f -> f.getName().equals(fieldName))
+        .findFirst();
+    if (field.isPresent()) {
+      return field.get().getAnnotation(Config.class);
+    }
+    return null;

Review comment:
       `findFieldConfigAnnotationByName` could return `Optional<Config>` instead of `Config` or `null`:
   
   ```java
           .findFirst()
           .map(field -> field.getAnnotation(Config.class));
   ```
   
   This would let the other new methods (`getType`, etc.) to be simplified to:
   
   ```java
       return findFieldConfigAnnotationByName(configClass, fieldName)
           .map(Config::type);
   ```

##########
File path: hadoop-hdds/config/src/test/java/org/apache/hadoop/hdds/conf/TestConfigurationReflectionUtil.java
##########
@@ -0,0 +1,71 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.conf;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.util.Optional;
+
+/**
+ * Test the configuration reflection utility class.
+ */
+public class TestConfigurationReflectionUtil {

Review comment:
       Can you please also add test cases for:
   
    * "field from parent class" (eg. `ConfigurationExample.class, "secure"`),
    * "non-existent field" (eg. `ConfigurationExample.class, "no-such-field"`), and
    * "field without `@Config` annotation" (eg. `ConfigFileAppender.class, "document"`)?

##########
File path: hadoop-hdds/config/src/test/java/org/apache/hadoop/hdds/conf/TestConfigurationReflectionUtil.java
##########
@@ -0,0 +1,71 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.conf;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.util.Optional;
+
+/**
+ * Test the configuration reflection utility class.
+ */
+public class TestConfigurationReflectionUtil {
+
+  @Test
+  public void testClassWithConfigGroup() {
+    Optional<ConfigType> actualType =
+        ConfigurationReflectionUtil.getType(
+            ConfigurationExample.class, "waitTime");
+    Assert.assertTrue(actualType.isPresent());
+    Assert.assertEquals(ConfigType.TIME, actualType.get());
+
+    Optional<String> actualKey =
+        ConfigurationReflectionUtil.getKey(
+        ConfigurationExample.class, "waitTime");
+    Assert.assertTrue(actualKey.isPresent());
+    Assert.assertEquals("ozone.scm.client.wait", actualKey.get());
+
+    Optional<String> actualDefaultValue =
+        ConfigurationReflectionUtil.getDefaultValue(
+            ConfigurationExample.class, "waitTime");
+    Assert.assertTrue(actualKey.isPresent());

Review comment:
       ```suggestion
       Assert.assertTrue(actualDefaultValue.isPresent());
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong commented on pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong commented on pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#issuecomment-684589260


   @elek Thanks to create the Java-based-Configuration framework, it is better than the hadoop configuration, but sometime, I want to get the default value or key name for a config variable, so I submit this PR, hope this can supply a way to use Java-based-Configuration framework conveniently.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong commented on pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong commented on pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#issuecomment-686870825


   @adoroszlai Thanks to trigger the CI check for me.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong commented on a change in pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong commented on a change in pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#discussion_r482599257



##########
File path: hadoop-ozone/dev-support/intellij/ozone-site.xml
##########
@@ -59,12 +59,12 @@
     <name>hdds.prometheus.endpoint.enabled</name>
     <value>true</value>
   </property>
-  <property>
-    <name>ozone.recon.address</name>
-    <value>localhost:9891</value>
-  </property>
-  <property>
-    <name>ozone.recon.db.dir</name>
-    <value>/tmp/recon</value>
-  </property>
+<!--  <property>-->
+<!--    <name>ozone.recon.address</name>-->
+<!--    <value>localhost:9891</value>-->
+<!--  </property>-->
+<!--  <property>-->
+<!--    <name>ozone.recon.db.dir</name>-->
+<!--    <value>/tmp/recon</value>-->
+<!--  </property>-->

Review comment:
       Sorry for that




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org

[GitHub] [hadoop-ozone] maobaolong commented on pull request #1369: HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily

Posted by GitBox <gi...@apache.org>.
maobaolong commented on pull request #1369:
URL: https://github.com/apache/hadoop-ozone/pull/1369#issuecomment-694162379


   @adoroszlai Thanks you merge this PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org