You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@amoro.apache.org by "Jzjsnow (via GitHub)" <gi...@apache.org> on 2025/01/06 09:03:17 UTC
[PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Jzjsnow opened a new pull request, #3396:
URL: https://github.com/apache/amoro/pull/3396
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://amoro.apache.org/how-to-contribute/
2. If the PR is related to an issue in https://github.com/apache/amoro/issues, add '[AMORO-XXXX]' in your PR title, e.g., '[AMORO-XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][AMORO-XXXX] Your PR title ...'.
-->
## Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about its use case.
2. If you fix a bug, you can clarify why it is a bug.
3. Use Fix/Resolve/Close #{ISSUE_NUMBER} to link this PR to its related issue
-->
When we start Amoro Management Service , we need to set configuration items in plaintext in the config file, including sensitive configurations such as admin-password and the passwords for connecting to databases (e.g., mysql, pg, etc.), which may be a security risk. To avoid the use of plaintext passwords, we provide an interface (`ConfigShade`) by implementing which developers can customize the decryption method themselves.
We also provide an implementation for base64 encoding first, not only as an example implementation of the interface, but also to solve the current problem of plaintext passwords.
Close #3335.
## Brief change log
<!--
Clearly describe the changes made in modules, classes, methods, etc.
-->
- Add interface `ConfigShade` to the `amoro-common` module to provide the ability to customize decryption of sensitive options in the config file
- Provide an implementation `org.apache.amoro.config.shade.impl.Base64ConfigShade` for base64 encoding
## How was this patch tested?
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] Run test locally before making a pull request
## Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "klion26 (via GitHub)" <gi...@apache.org>.
klion26 commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1906328416
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/utils/ConfigShadeUtils.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade.utils;
+
+import org.apache.amoro.config.ConfigOption;
+import org.apache.amoro.config.ConfigOptions;
+import org.apache.amoro.config.Configurations;
+import org.apache.amoro.config.shade.ConfigShade;
+import org.apache.amoro.shade.guava32.com.google.common.annotations.VisibleForTesting;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+import java.util.function.BiFunction;
+
+/** Config shade utilities. */
+public final class ConfigShadeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(ConfigShadeUtils.class);
+
+ private static final Map<String, ConfigShade> CONFIG_SHADES = new HashMap<>();
+
+ private static final ConfigShade DEFAULT_SHADE = new DefaultConfigShade();
+
+ public static final ConfigOption<String> SHADE_IDENTIFIER_OPTION =
+ ConfigOptions.key("shade.identifier")
+ .stringType()
+ .defaultValue("default")
+ .withDescription(
+ "The identifier of the encryption method for decryption. Defaults to \"default\", indicating no encryption");
+
+ public static final ConfigOption<List<String>> SHADE_SENSITIVE_KEYWORDS =
+ ConfigOptions.key("shade.sensitive-keywords")
+ .stringType()
+ .asList()
+ .defaultValues("admin-password", "database.password")
+ .withDescription(
+ "A semicolon-separated list of keywords for the configuration items to be decrypted.");
+
+ static {
+ ServiceLoader<ConfigShade> serviceLoader = ServiceLoader.load(ConfigShade.class);
+ Iterator<ConfigShade> it = serviceLoader.iterator();
+ it.forEachRemaining(
+ configShade -> {
+ CONFIG_SHADES.put(configShade.getIdentifier(), configShade);
+ });
+ LOG.info("Load config shade spi: {}", CONFIG_SHADES.keySet());
+ }
+
+ @VisibleForTesting
+ public static String decryptOption(String identifier, String content) {
+ ConfigShade configShade = CONFIG_SHADES.getOrDefault(identifier, DEFAULT_SHADE);
+ return configShade.decrypt(content);
+ }
+
+ public static Map<String, Object> decryptConfig(Map<String, Object> configMap) throws Exception {
+ Configurations serviceConfig = Configurations.fromObjectMap(configMap);
+ String identifier = serviceConfig.get(SHADE_IDENTIFIER_OPTION);
+ List<String> sensitiveOptions = serviceConfig.get(SHADE_SENSITIVE_KEYWORDS);
+ return decryptConfig(identifier, configMap, sensitiveOptions, serviceConfig);
+ }
+
+ @SuppressWarnings("unchecked")
+ public static Map<String, Object> decryptConfig(
+ String identifier,
+ Map<String, Object> configMap,
+ List<String> sensitiveOptions,
+ Configurations serviceConfig)
+ throws Exception {
+ ConfigShade configShade = CONFIG_SHADES.get(identifier);
+ if (configShade == null) {
+ LOG.error("Can not find config shade: {}", identifier);
+ throw new IllegalStateException("Can not find config shade: " + identifier);
+ }
+ configShade.initialize(serviceConfig);
+
+ if (DEFAULT_SHADE.getIdentifier().equals(configShade.getIdentifier())) {
+ return configMap;
+ }
+
+ LOG.info("Use config shade: {}", identifier);
+ BiFunction<String, Object, String> processFunction =
+ (key, value) -> configShade.decrypt(value.toString());
+
+ Preconditions.checkArgument(
+ !configMap.isEmpty(), "Miss ams config! Please check the config file.");
+
+ for (String sensitiveOption : sensitiveOptions) {
+ try {
+ configMap.computeIfPresent(sensitiveOption, processFunction);
+ } catch (Exception e) {
+ LOG.error("Failed to decrypt sensitive option {}:", sensitiveOption, e);
Review Comment:
do we need to throw exception here if no key found
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "Jzjsnow (via GitHub)" <gi...@apache.org>.
Jzjsnow commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1906972668
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/utils/ConfigShadeUtils.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade.utils;
+
+import org.apache.amoro.config.ConfigOption;
+import org.apache.amoro.config.ConfigOptions;
+import org.apache.amoro.config.Configurations;
+import org.apache.amoro.config.shade.ConfigShade;
+import org.apache.amoro.shade.guava32.com.google.common.annotations.VisibleForTesting;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+import java.util.function.BiFunction;
+
+/** Config shade utilities. */
+public final class ConfigShadeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(ConfigShadeUtils.class);
+
+ private static final Map<String, ConfigShade> CONFIG_SHADES = new HashMap<>();
+
+ private static final ConfigShade DEFAULT_SHADE = new DefaultConfigShade();
+
+ public static final ConfigOption<String> SHADE_IDENTIFIER_OPTION =
+ ConfigOptions.key("shade.identifier")
+ .stringType()
+ .defaultValue("default")
+ .withDescription(
+ "The identifier of the encryption method for decryption. Defaults to \"default\", indicating no encryption");
+
+ public static final ConfigOption<List<String>> SHADE_SENSITIVE_KEYWORDS =
+ ConfigOptions.key("shade.sensitive-keywords")
+ .stringType()
+ .asList()
+ .defaultValues("admin-password", "database.password")
+ .withDescription(
+ "A semicolon-separated list of keywords for the configuration items to be decrypted.");
+
+ static {
+ ServiceLoader<ConfigShade> serviceLoader = ServiceLoader.load(ConfigShade.class);
+ Iterator<ConfigShade> it = serviceLoader.iterator();
+ it.forEachRemaining(
+ configShade -> {
+ CONFIG_SHADES.put(configShade.getIdentifier(), configShade);
+ });
+ LOG.info("Load config shade spi: {}", CONFIG_SHADES.keySet());
+ }
+
+ @VisibleForTesting
+ public static String decryptOption(String identifier, String content) {
+ ConfigShade configShade = CONFIG_SHADES.getOrDefault(identifier, DEFAULT_SHADE);
Review Comment:
Yes, it is necessary to print the log. The method `decryptConfig` is called for normal use, and if the given identifier cannot be loaded, it will log ERROR and throw an IllegalStateException exception. Here, the method `decryptOption` is only used for unit testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "Jzjsnow (via GitHub)" <gi...@apache.org>.
Jzjsnow commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1907078375
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/utils/ConfigShadeUtils.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade.utils;
+
+import org.apache.amoro.config.ConfigOption;
+import org.apache.amoro.config.ConfigOptions;
+import org.apache.amoro.config.Configurations;
+import org.apache.amoro.config.shade.ConfigShade;
+import org.apache.amoro.shade.guava32.com.google.common.annotations.VisibleForTesting;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+import java.util.function.BiFunction;
+
+/** Config shade utilities. */
+public final class ConfigShadeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(ConfigShadeUtils.class);
+
+ private static final Map<String, ConfigShade> CONFIG_SHADES = new HashMap<>();
+
+ private static final ConfigShade DEFAULT_SHADE = new DefaultConfigShade();
+
+ public static final ConfigOption<String> SHADE_IDENTIFIER_OPTION =
+ ConfigOptions.key("shade.identifier")
+ .stringType()
+ .defaultValue("default")
+ .withDescription(
+ "The identifier of the encryption method for decryption. Defaults to \"default\", indicating no encryption");
+
+ public static final ConfigOption<List<String>> SHADE_SENSITIVE_KEYWORDS =
+ ConfigOptions.key("shade.sensitive-keywords")
+ .stringType()
+ .asList()
+ .defaultValues("admin-password", "database.password")
Review Comment:
Currently, the sensitive keywords are all in the `ams` part, so first I put the shade-related configurations under `ams` part and only decrypt the `ams` configurations . If in the future the containers also contain sensitive information, I think it may be necessary to put the shade-related configurations in a separate part, alongside the `ams` and `containers`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #3396:
URL: https://github.com/apache/amoro/pull/3396#issuecomment-2579138793
## [Codecov](https://app.codecov.io/gh/apache/amoro/pull/3396?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `0%` with `59 lines` in your changes missing coverage. Please review.
> Project coverage is 21.59%. Comparing base [(`243d289`)](https://app.codecov.io/gh/apache/amoro/commit/243d2894c6bbba59b74be97a1ea93f935f0bd9ad?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`67c9d3e`)](https://app.codecov.io/gh/apache/amoro/commit/67c9d3e33669c55190e5a97161f3352ade1ef839?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/amoro/pull/3396?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...che/amoro/config/shade/utils/ConfigShadeUtils.java](https://app.codecov.io/gh/apache/amoro/pull/3396?src=pr&el=tree&filepath=amoro-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Famoro%2Fconfig%2Fshade%2Futils%2FConfigShadeUtils.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YW1vcm8tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9hbW9yby9jb25maWcvc2hhZGUvdXRpbHMvQ29uZmlnU2hhZGVVdGlscy5qYXZh) | 0.00% | [54 Missing :warning: ](https://app.codecov.io/gh/apache/amoro/pull/3396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...che/amoro/config/shade/impl/Base64ConfigShade.java](https://app.codecov.io/gh/apache/amoro/pull/3396?src=pr&el=tree&filepath=amoro-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Famoro%2Fconfig%2Fshade%2Fimpl%2FBase64ConfigShade.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YW1vcm8tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9hbW9yby9jb25maWcvc2hhZGUvaW1wbC9CYXNlNjRDb25maWdTaGFkZS5qYXZh) | 0.00% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/amoro/pull/3396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ava/org/apache/amoro/config/shade/ConfigShade.java](https://app.codecov.io/gh/apache/amoro/pull/3396?src=pr&el=tree&filepath=amoro-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Famoro%2Fconfig%2Fshade%2FConfigShade.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YW1vcm8tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9hbW9yby9jb25maWcvc2hhZGUvQ29uZmlnU2hhZGUuamF2YQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/amoro/pull/3396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #3396 +/- ##
============================================
- Coverage 21.59% 21.59% -0.01%
- Complexity 2309 2313 +4
============================================
Files 426 429 +3
Lines 39719 39791 +72
Branches 5624 5632 +8
============================================
+ Hits 8577 8591 +14
- Misses 30414 30473 +59
+ Partials 728 727 -1
```
| [Flag](https://app.codecov.io/gh/apache/amoro/pull/3396/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [trino](https://app.codecov.io/gh/apache/amoro/pull/3396/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `21.59% <0.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/amoro/pull/3396?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "zhoujinsong (via GitHub)" <gi...@apache.org>.
zhoujinsong commented on PR #3396:
URL: https://github.com/apache/amoro/pull/3396#issuecomment-2609016827
Hi,
Thanks for the contribution!
Can you help to add the configuration example into the `dist/src/main/amoro-bin/conf/config.yaml`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "Jzjsnow (via GitHub)" <gi...@apache.org>.
Jzjsnow commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1907055452
##########
amoro-ams/src/test/java/org/apache/amoro/server/config/TestConfigShade.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.server.config;
+
+import static org.apache.amoro.server.AmoroServiceContainer.expandConfigMap;
+
+import org.apache.amoro.config.shade.utils.ConfigShadeUtils;
+import org.apache.amoro.server.AmoroManagementConf;
+import org.apache.amoro.shade.guava32.com.google.common.collect.ImmutableMap;
+import org.apache.amoro.shade.guava32.com.google.common.collect.Maps;
+import org.apache.amoro.shade.guava32.com.google.common.io.Resources;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.core.type.TypeReference;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.databind.JsonNode;
+import org.apache.amoro.utils.JacksonUtil;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.yaml.snakeyaml.Yaml;
+
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Map;
+
+public class TestConfigShade {
+ private static final String USERNAME = "admin";
+ private static final String PASSWORD = "password";
+
+ @Test
+ public void testDecryptOptions() {
+ String encryptUsername = "YWRtaW4=";
Review Comment:
In the new commit, `TestConfigShade#getBase64EncodedText` is provided in the unit test to show the base64 encoding process.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "klion26 (via GitHub)" <gi...@apache.org>.
klion26 commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1906328416
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/utils/ConfigShadeUtils.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade.utils;
+
+import org.apache.amoro.config.ConfigOption;
+import org.apache.amoro.config.ConfigOptions;
+import org.apache.amoro.config.Configurations;
+import org.apache.amoro.config.shade.ConfigShade;
+import org.apache.amoro.shade.guava32.com.google.common.annotations.VisibleForTesting;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+import java.util.function.BiFunction;
+
+/** Config shade utilities. */
+public final class ConfigShadeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(ConfigShadeUtils.class);
+
+ private static final Map<String, ConfigShade> CONFIG_SHADES = new HashMap<>();
+
+ private static final ConfigShade DEFAULT_SHADE = new DefaultConfigShade();
+
+ public static final ConfigOption<String> SHADE_IDENTIFIER_OPTION =
+ ConfigOptions.key("shade.identifier")
+ .stringType()
+ .defaultValue("default")
+ .withDescription(
+ "The identifier of the encryption method for decryption. Defaults to \"default\", indicating no encryption");
+
+ public static final ConfigOption<List<String>> SHADE_SENSITIVE_KEYWORDS =
+ ConfigOptions.key("shade.sensitive-keywords")
+ .stringType()
+ .asList()
+ .defaultValues("admin-password", "database.password")
+ .withDescription(
+ "A semicolon-separated list of keywords for the configuration items to be decrypted.");
+
+ static {
+ ServiceLoader<ConfigShade> serviceLoader = ServiceLoader.load(ConfigShade.class);
+ Iterator<ConfigShade> it = serviceLoader.iterator();
+ it.forEachRemaining(
+ configShade -> {
+ CONFIG_SHADES.put(configShade.getIdentifier(), configShade);
+ });
+ LOG.info("Load config shade spi: {}", CONFIG_SHADES.keySet());
+ }
+
+ @VisibleForTesting
+ public static String decryptOption(String identifier, String content) {
+ ConfigShade configShade = CONFIG_SHADES.getOrDefault(identifier, DEFAULT_SHADE);
+ return configShade.decrypt(content);
+ }
+
+ public static Map<String, Object> decryptConfig(Map<String, Object> configMap) throws Exception {
+ Configurations serviceConfig = Configurations.fromObjectMap(configMap);
+ String identifier = serviceConfig.get(SHADE_IDENTIFIER_OPTION);
+ List<String> sensitiveOptions = serviceConfig.get(SHADE_SENSITIVE_KEYWORDS);
+ return decryptConfig(identifier, configMap, sensitiveOptions, serviceConfig);
+ }
+
+ @SuppressWarnings("unchecked")
+ public static Map<String, Object> decryptConfig(
+ String identifier,
+ Map<String, Object> configMap,
+ List<String> sensitiveOptions,
+ Configurations serviceConfig)
+ throws Exception {
+ ConfigShade configShade = CONFIG_SHADES.get(identifier);
+ if (configShade == null) {
+ LOG.error("Can not find config shade: {}", identifier);
+ throw new IllegalStateException("Can not find config shade: " + identifier);
+ }
+ configShade.initialize(serviceConfig);
+
+ if (DEFAULT_SHADE.getIdentifier().equals(configShade.getIdentifier())) {
+ return configMap;
+ }
+
+ LOG.info("Use config shade: {}", identifier);
+ BiFunction<String, Object, String> processFunction =
+ (key, value) -> configShade.decrypt(value.toString());
+
+ Preconditions.checkArgument(
+ !configMap.isEmpty(), "Miss ams config! Please check the config file.");
+
+ for (String sensitiveOption : sensitiveOptions) {
+ try {
+ configMap.computeIfPresent(sensitiveOption, processFunction);
+ } catch (Exception e) {
+ LOG.error("Failed to decrypt sensitive option {}:", sensitiveOption, e);
Review Comment:
do we need to throw exception here if no key found
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "Jzjsnow (via GitHub)" <gi...@apache.org>.
Jzjsnow commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1907052056
##########
amoro-ams/src/test/java/org/apache/amoro/server/config/TestConfigShade.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.server.config;
+
+import static org.apache.amoro.server.AmoroServiceContainer.expandConfigMap;
+
+import org.apache.amoro.config.shade.utils.ConfigShadeUtils;
+import org.apache.amoro.server.AmoroManagementConf;
+import org.apache.amoro.shade.guava32.com.google.common.collect.ImmutableMap;
+import org.apache.amoro.shade.guava32.com.google.common.collect.Maps;
+import org.apache.amoro.shade.guava32.com.google.common.io.Resources;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.core.type.TypeReference;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.databind.JsonNode;
+import org.apache.amoro.utils.JacksonUtil;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.yaml.snakeyaml.Yaml;
+
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Map;
+
+public class TestConfigShade {
+ private static final String USERNAME = "admin";
+ private static final String PASSWORD = "password";
+
+ @Test
+ public void testDecryptOptions() {
+ String encryptUsername = "YWRtaW4=";
+ String encryptPassword = "cGFzc3dvcmQ=";
+ String decryptUsername = ConfigShadeUtils.decryptOption("base64", encryptUsername);
Review Comment:
I have changed the identifier string to a static final string with the value of `Base64ConfigShade.getIdentifier()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "Jzjsnow (via GitHub)" <gi...@apache.org>.
Jzjsnow commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1907062494
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/utils/ConfigShadeUtils.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade.utils;
+
+import org.apache.amoro.config.ConfigOption;
+import org.apache.amoro.config.ConfigOptions;
+import org.apache.amoro.config.Configurations;
+import org.apache.amoro.config.shade.ConfigShade;
+import org.apache.amoro.shade.guava32.com.google.common.annotations.VisibleForTesting;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+import java.util.function.BiFunction;
+
+/** Config shade utilities. */
+public final class ConfigShadeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(ConfigShadeUtils.class);
+
+ private static final Map<String, ConfigShade> CONFIG_SHADES = new HashMap<>();
+
+ private static final ConfigShade DEFAULT_SHADE = new DefaultConfigShade();
+
+ public static final ConfigOption<String> SHADE_IDENTIFIER_OPTION =
+ ConfigOptions.key("shade.identifier")
+ .stringType()
+ .defaultValue("default")
+ .withDescription(
+ "The identifier of the encryption method for decryption. Defaults to \"default\", indicating no encryption");
+
+ public static final ConfigOption<List<String>> SHADE_SENSITIVE_KEYWORDS =
+ ConfigOptions.key("shade.sensitive-keywords")
+ .stringType()
+ .asList()
+ .defaultValues("admin-password", "database.password")
+ .withDescription(
+ "A semicolon-separated list of keywords for the configuration items to be decrypted.");
+
+ static {
+ ServiceLoader<ConfigShade> serviceLoader = ServiceLoader.load(ConfigShade.class);
+ Iterator<ConfigShade> it = serviceLoader.iterator();
+ it.forEachRemaining(
+ configShade -> {
+ CONFIG_SHADES.put(configShade.getIdentifier(), configShade);
Review Comment:
I add the log to indicate the identifier of the `ConfigShade` implementation and the full name of the class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "Jzjsnow (via GitHub)" <gi...@apache.org>.
Jzjsnow commented on PR #3396:
URL: https://github.com/apache/amoro/pull/3396#issuecomment-2609124940
@zhoujinsong Thanks for the reminder! I've added the relevant parameters to the template `config.yaml` in the latest commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "Jzjsnow (via GitHub)" <gi...@apache.org>.
Jzjsnow commented on PR #3396:
URL: https://github.com/apache/amoro/pull/3396#issuecomment-2606787809
@czy006 Thanks for the heads up, I've changed the script in helm charts in the latest commit and verified it locally:
Similar to the usage in `config.yaml`, users needs to fill in the options `amoroConf.shade.identifier` and `amoroConf.shade.sensitiveKeywords` in `charts/amoro/values.yaml` and replace the plaintext of the sensitive items specified in `sensitive-keywords` with the encrypted ciphertext. Then deploy it via the `helm install` command to make it work, as described in the official documentation.
If users want to use their customized encryption algorithm, they need to modify the amoro image prior to the above steps, as they need to upload their decryption dependency package to the image first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "Jzjsnow (via GitHub)" <gi...@apache.org>.
Jzjsnow commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1907040061
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/utils/ConfigShadeUtils.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade.utils;
+
+import org.apache.amoro.config.ConfigOption;
+import org.apache.amoro.config.ConfigOptions;
+import org.apache.amoro.config.Configurations;
+import org.apache.amoro.config.shade.ConfigShade;
+import org.apache.amoro.shade.guava32.com.google.common.annotations.VisibleForTesting;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+import java.util.function.BiFunction;
+
+/** Config shade utilities. */
+public final class ConfigShadeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(ConfigShadeUtils.class);
+
+ private static final Map<String, ConfigShade> CONFIG_SHADES = new HashMap<>();
+
+ private static final ConfigShade DEFAULT_SHADE = new DefaultConfigShade();
+
+ public static final ConfigOption<String> SHADE_IDENTIFIER_OPTION =
+ ConfigOptions.key("shade.identifier")
+ .stringType()
+ .defaultValue("default")
+ .withDescription(
+ "The identifier of the encryption method for decryption. Defaults to \"default\", indicating no encryption");
+
+ public static final ConfigOption<List<String>> SHADE_SENSITIVE_KEYWORDS =
+ ConfigOptions.key("shade.sensitive-keywords")
+ .stringType()
+ .asList()
+ .defaultValues("admin-password", "database.password")
+ .withDescription(
+ "A semicolon-separated list of keywords for the configuration items to be decrypted.");
+
+ static {
+ ServiceLoader<ConfigShade> serviceLoader = ServiceLoader.load(ConfigShade.class);
+ Iterator<ConfigShade> it = serviceLoader.iterator();
+ it.forEachRemaining(
+ configShade -> {
+ CONFIG_SHADES.put(configShade.getIdentifier(), configShade);
+ });
+ LOG.info("Load config shade spi: {}", CONFIG_SHADES.keySet());
+ }
+
+ @VisibleForTesting
+ public static String decryptOption(String identifier, String content) {
+ ConfigShade configShade = CONFIG_SHADES.getOrDefault(identifier, DEFAULT_SHADE);
+ return configShade.decrypt(content);
+ }
+
+ public static Map<String, Object> decryptConfig(Map<String, Object> configMap) throws Exception {
+ Configurations serviceConfig = Configurations.fromObjectMap(configMap);
+ String identifier = serviceConfig.get(SHADE_IDENTIFIER_OPTION);
+ List<String> sensitiveOptions = serviceConfig.get(SHADE_SENSITIVE_KEYWORDS);
+ return decryptConfig(identifier, configMap, sensitiveOptions, serviceConfig);
+ }
+
+ @SuppressWarnings("unchecked")
+ public static Map<String, Object> decryptConfig(
+ String identifier,
+ Map<String, Object> configMap,
+ List<String> sensitiveOptions,
+ Configurations serviceConfig)
+ throws Exception {
+ ConfigShade configShade = CONFIG_SHADES.get(identifier);
+ if (configShade == null) {
+ LOG.error("Can not find config shade: {}", identifier);
+ throw new IllegalStateException("Can not find config shade: " + identifier);
+ }
+ configShade.initialize(serviceConfig);
+
+ if (DEFAULT_SHADE.getIdentifier().equals(configShade.getIdentifier())) {
+ return configMap;
+ }
+
+ LOG.info("Use config shade: {}", identifier);
+ BiFunction<String, Object, String> processFunction =
+ (key, value) -> configShade.decrypt(value.toString());
+
+ Preconditions.checkArgument(
Review Comment:
Here we can skip checking for an empty map, as it will be returned ahead of time when the configshade is recognized as the default value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "Jzjsnow (via GitHub)" <gi...@apache.org>.
Jzjsnow commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1907055452
##########
amoro-ams/src/test/java/org/apache/amoro/server/config/TestConfigShade.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.server.config;
+
+import static org.apache.amoro.server.AmoroServiceContainer.expandConfigMap;
+
+import org.apache.amoro.config.shade.utils.ConfigShadeUtils;
+import org.apache.amoro.server.AmoroManagementConf;
+import org.apache.amoro.shade.guava32.com.google.common.collect.ImmutableMap;
+import org.apache.amoro.shade.guava32.com.google.common.collect.Maps;
+import org.apache.amoro.shade.guava32.com.google.common.io.Resources;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.core.type.TypeReference;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.databind.JsonNode;
+import org.apache.amoro.utils.JacksonUtil;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.yaml.snakeyaml.Yaml;
+
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Map;
+
+public class TestConfigShade {
+ private static final String USERNAME = "admin";
+ private static final String PASSWORD = "password";
+
+ @Test
+ public void testDecryptOptions() {
+ String encryptUsername = "YWRtaW4=";
Review Comment:
In the new commit, `TestConfigShade#getBase64EncodedText` is provided to show the base64 encoding process and can be used to encode plaintext for testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "klion26 (via GitHub)" <gi...@apache.org>.
klion26 merged PR #3396:
URL: https://github.com/apache/amoro/pull/3396
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "klion26 (via GitHub)" <gi...@apache.org>.
klion26 commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1906266934
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/utils/ConfigShadeUtils.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade.utils;
+
+import org.apache.amoro.config.ConfigOption;
+import org.apache.amoro.config.ConfigOptions;
+import org.apache.amoro.config.Configurations;
+import org.apache.amoro.config.shade.ConfigShade;
+import org.apache.amoro.shade.guava32.com.google.common.annotations.VisibleForTesting;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+import java.util.function.BiFunction;
+
+/** Config shade utilities. */
+public final class ConfigShadeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(ConfigShadeUtils.class);
+
+ private static final Map<String, ConfigShade> CONFIG_SHADES = new HashMap<>();
+
+ private static final ConfigShade DEFAULT_SHADE = new DefaultConfigShade();
+
+ public static final ConfigOption<String> SHADE_IDENTIFIER_OPTION =
+ ConfigOptions.key("shade.identifier")
+ .stringType()
+ .defaultValue("default")
+ .withDescription(
+ "The identifier of the encryption method for decryption. Defaults to \"default\", indicating no encryption");
+
+ public static final ConfigOption<List<String>> SHADE_SENSITIVE_KEYWORDS =
+ ConfigOptions.key("shade.sensitive-keywords")
+ .stringType()
+ .asList()
+ .defaultValues("admin-password", "database.password")
+ .withDescription(
+ "A semicolon-separated list of keywords for the configuration items to be decrypted.");
+
+ static {
+ ServiceLoader<ConfigShade> serviceLoader = ServiceLoader.load(ConfigShade.class);
+ Iterator<ConfigShade> it = serviceLoader.iterator();
+ it.forEachRemaining(
+ configShade -> {
+ CONFIG_SHADES.put(configShade.getIdentifier(), configShade);
+ });
+ LOG.info("Load config shade spi: {}", CONFIG_SHADES.keySet());
+ }
+
+ @VisibleForTesting
+ public static String decryptOption(String identifier, String content) {
+ ConfigShade configShade = CONFIG_SHADES.getOrDefault(identifier, DEFAULT_SHADE);
+ return configShade.decrypt(content);
+ }
+
+ public static Map<String, Object> decryptConfig(Map<String, Object> configMap) throws Exception {
+ Configurations serviceConfig = Configurations.fromObjectMap(configMap);
+ String identifier = serviceConfig.get(SHADE_IDENTIFIER_OPTION);
+ List<String> sensitiveOptions = serviceConfig.get(SHADE_SENSITIVE_KEYWORDS);
+ return decryptConfig(identifier, configMap, sensitiveOptions, serviceConfig);
+ }
+
+ @SuppressWarnings("unchecked")
Review Comment:
this annotation can be removed
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/utils/ConfigShadeUtils.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade.utils;
+
+import org.apache.amoro.config.ConfigOption;
+import org.apache.amoro.config.ConfigOptions;
+import org.apache.amoro.config.Configurations;
+import org.apache.amoro.config.shade.ConfigShade;
+import org.apache.amoro.shade.guava32.com.google.common.annotations.VisibleForTesting;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+import java.util.function.BiFunction;
+
+/** Config shade utilities. */
+public final class ConfigShadeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(ConfigShadeUtils.class);
+
+ private static final Map<String, ConfigShade> CONFIG_SHADES = new HashMap<>();
+
+ private static final ConfigShade DEFAULT_SHADE = new DefaultConfigShade();
+
+ public static final ConfigOption<String> SHADE_IDENTIFIER_OPTION =
+ ConfigOptions.key("shade.identifier")
+ .stringType()
+ .defaultValue("default")
+ .withDescription(
+ "The identifier of the encryption method for decryption. Defaults to \"default\", indicating no encryption");
+
+ public static final ConfigOption<List<String>> SHADE_SENSITIVE_KEYWORDS =
+ ConfigOptions.key("shade.sensitive-keywords")
+ .stringType()
+ .asList()
+ .defaultValues("admin-password", "database.password")
Review Comment:
seems the the key here is only located in `ams` part in the configuration, not sure if this needs to be apply the whole config
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/utils/ConfigShadeUtils.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade.utils;
+
+import org.apache.amoro.config.ConfigOption;
+import org.apache.amoro.config.ConfigOptions;
+import org.apache.amoro.config.Configurations;
+import org.apache.amoro.config.shade.ConfigShade;
+import org.apache.amoro.shade.guava32.com.google.common.annotations.VisibleForTesting;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+import java.util.function.BiFunction;
+
+/** Config shade utilities. */
+public final class ConfigShadeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(ConfigShadeUtils.class);
+
+ private static final Map<String, ConfigShade> CONFIG_SHADES = new HashMap<>();
+
+ private static final ConfigShade DEFAULT_SHADE = new DefaultConfigShade();
+
+ public static final ConfigOption<String> SHADE_IDENTIFIER_OPTION =
+ ConfigOptions.key("shade.identifier")
+ .stringType()
+ .defaultValue("default")
+ .withDescription(
+ "The identifier of the encryption method for decryption. Defaults to \"default\", indicating no encryption");
+
+ public static final ConfigOption<List<String>> SHADE_SENSITIVE_KEYWORDS =
+ ConfigOptions.key("shade.sensitive-keywords")
+ .stringType()
+ .asList()
+ .defaultValues("admin-password", "database.password")
+ .withDescription(
+ "A semicolon-separated list of keywords for the configuration items to be decrypted.");
+
+ static {
+ ServiceLoader<ConfigShade> serviceLoader = ServiceLoader.load(ConfigShade.class);
+ Iterator<ConfigShade> it = serviceLoader.iterator();
+ it.forEachRemaining(
+ configShade -> {
+ CONFIG_SHADES.put(configShade.getIdentifier(), configShade);
Review Comment:
Do we need to log the relationship for the identifier and the configShade here? I assume the shade loaded here would not a big number.
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/ConfigShade.java:
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade;
+
+import org.apache.amoro.config.Configurations;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * The interface that provides the ability to decrypt {@link
+ * org.apache.amoro.config.Configurations}.
+ */
+public interface ConfigShade {
+ Logger LOG = LoggerFactory.getLogger(ConfigShade.class);
Review Comment:
seems the `LOG` here is no more used
##########
amoro-ams/src/test/java/org/apache/amoro/server/config/TestConfigShade.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.server.config;
+
+import static org.apache.amoro.server.AmoroServiceContainer.expandConfigMap;
+
+import org.apache.amoro.config.shade.utils.ConfigShadeUtils;
+import org.apache.amoro.server.AmoroManagementConf;
+import org.apache.amoro.shade.guava32.com.google.common.collect.ImmutableMap;
+import org.apache.amoro.shade.guava32.com.google.common.collect.Maps;
+import org.apache.amoro.shade.guava32.com.google.common.io.Resources;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.core.type.TypeReference;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.databind.JsonNode;
+import org.apache.amoro.utils.JacksonUtil;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.yaml.snakeyaml.Yaml;
+
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Map;
+
+public class TestConfigShade {
+ private static final String USERNAME = "admin";
+ private static final String PASSWORD = "password";
+
+ @Test
+ public void testDecryptOptions() {
+ String encryptUsername = "YWRtaW4=";
Review Comment:
we could add some comments here to show that these are the message have been transformed by base64
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/utils/ConfigShadeUtils.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade.utils;
+
+import org.apache.amoro.config.ConfigOption;
+import org.apache.amoro.config.ConfigOptions;
+import org.apache.amoro.config.Configurations;
+import org.apache.amoro.config.shade.ConfigShade;
+import org.apache.amoro.shade.guava32.com.google.common.annotations.VisibleForTesting;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+import java.util.function.BiFunction;
+
+/** Config shade utilities. */
+public final class ConfigShadeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(ConfigShadeUtils.class);
+
+ private static final Map<String, ConfigShade> CONFIG_SHADES = new HashMap<>();
+
+ private static final ConfigShade DEFAULT_SHADE = new DefaultConfigShade();
+
+ public static final ConfigOption<String> SHADE_IDENTIFIER_OPTION =
+ ConfigOptions.key("shade.identifier")
+ .stringType()
+ .defaultValue("default")
+ .withDescription(
+ "The identifier of the encryption method for decryption. Defaults to \"default\", indicating no encryption");
+
+ public static final ConfigOption<List<String>> SHADE_SENSITIVE_KEYWORDS =
+ ConfigOptions.key("shade.sensitive-keywords")
+ .stringType()
+ .asList()
+ .defaultValues("admin-password", "database.password")
+ .withDescription(
+ "A semicolon-separated list of keywords for the configuration items to be decrypted.");
+
+ static {
+ ServiceLoader<ConfigShade> serviceLoader = ServiceLoader.load(ConfigShade.class);
+ Iterator<ConfigShade> it = serviceLoader.iterator();
+ it.forEachRemaining(
+ configShade -> {
+ CONFIG_SHADES.put(configShade.getIdentifier(), configShade);
+ });
+ LOG.info("Load config shade spi: {}", CONFIG_SHADES.keySet());
+ }
+
+ @VisibleForTesting
+ public static String decryptOption(String identifier, String content) {
+ ConfigShade configShade = CONFIG_SHADES.getOrDefault(identifier, DEFAULT_SHADE);
Review Comment:
do we need to add some logs here if there is no `ConfigShade` with the given `identifier`
##########
amoro-common/src/main/java/org/apache/amoro/config/shade/utils/ConfigShadeUtils.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.config.shade.utils;
+
+import org.apache.amoro.config.ConfigOption;
+import org.apache.amoro.config.ConfigOptions;
+import org.apache.amoro.config.Configurations;
+import org.apache.amoro.config.shade.ConfigShade;
+import org.apache.amoro.shade.guava32.com.google.common.annotations.VisibleForTesting;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.ServiceLoader;
+import java.util.function.BiFunction;
+
+/** Config shade utilities. */
+public final class ConfigShadeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(ConfigShadeUtils.class);
+
+ private static final Map<String, ConfigShade> CONFIG_SHADES = new HashMap<>();
+
+ private static final ConfigShade DEFAULT_SHADE = new DefaultConfigShade();
+
+ public static final ConfigOption<String> SHADE_IDENTIFIER_OPTION =
+ ConfigOptions.key("shade.identifier")
+ .stringType()
+ .defaultValue("default")
+ .withDescription(
+ "The identifier of the encryption method for decryption. Defaults to \"default\", indicating no encryption");
+
+ public static final ConfigOption<List<String>> SHADE_SENSITIVE_KEYWORDS =
+ ConfigOptions.key("shade.sensitive-keywords")
+ .stringType()
+ .asList()
+ .defaultValues("admin-password", "database.password")
+ .withDescription(
+ "A semicolon-separated list of keywords for the configuration items to be decrypted.");
+
+ static {
+ ServiceLoader<ConfigShade> serviceLoader = ServiceLoader.load(ConfigShade.class);
+ Iterator<ConfigShade> it = serviceLoader.iterator();
+ it.forEachRemaining(
+ configShade -> {
+ CONFIG_SHADES.put(configShade.getIdentifier(), configShade);
+ });
+ LOG.info("Load config shade spi: {}", CONFIG_SHADES.keySet());
+ }
+
+ @VisibleForTesting
+ public static String decryptOption(String identifier, String content) {
+ ConfigShade configShade = CONFIG_SHADES.getOrDefault(identifier, DEFAULT_SHADE);
+ return configShade.decrypt(content);
+ }
+
+ public static Map<String, Object> decryptConfig(Map<String, Object> configMap) throws Exception {
+ Configurations serviceConfig = Configurations.fromObjectMap(configMap);
+ String identifier = serviceConfig.get(SHADE_IDENTIFIER_OPTION);
+ List<String> sensitiveOptions = serviceConfig.get(SHADE_SENSITIVE_KEYWORDS);
+ return decryptConfig(identifier, configMap, sensitiveOptions, serviceConfig);
+ }
+
+ @SuppressWarnings("unchecked")
+ public static Map<String, Object> decryptConfig(
+ String identifier,
+ Map<String, Object> configMap,
+ List<String> sensitiveOptions,
+ Configurations serviceConfig)
+ throws Exception {
+ ConfigShade configShade = CONFIG_SHADES.get(identifier);
+ if (configShade == null) {
+ LOG.error("Can not find config shade: {}", identifier);
+ throw new IllegalStateException("Can not find config shade: " + identifier);
+ }
+ configShade.initialize(serviceConfig);
+
+ if (DEFAULT_SHADE.getIdentifier().equals(configShade.getIdentifier())) {
+ return configMap;
+ }
+
+ LOG.info("Use config shade: {}", identifier);
+ BiFunction<String, Object, String> processFunction =
+ (key, value) -> configShade.decrypt(value.toString());
+
+ Preconditions.checkArgument(
Review Comment:
why do we need to check this here
##########
amoro-ams/src/test/java/org/apache/amoro/server/config/TestConfigShade.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.server.config;
+
+import static org.apache.amoro.server.AmoroServiceContainer.expandConfigMap;
+
+import org.apache.amoro.config.shade.utils.ConfigShadeUtils;
+import org.apache.amoro.server.AmoroManagementConf;
+import org.apache.amoro.shade.guava32.com.google.common.collect.ImmutableMap;
+import org.apache.amoro.shade.guava32.com.google.common.collect.Maps;
+import org.apache.amoro.shade.guava32.com.google.common.io.Resources;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.core.type.TypeReference;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.databind.JsonNode;
+import org.apache.amoro.utils.JacksonUtil;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.yaml.snakeyaml.Yaml;
+
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Map;
+
+public class TestConfigShade {
+ private static final String USERNAME = "admin";
+ private static final String PASSWORD = "password";
+
+ @Test
+ public void testDecryptOptions() {
+ String encryptUsername = "YWRtaW4=";
+ String encryptPassword = "cGFzc3dvcmQ=";
+ String decryptUsername = ConfigShadeUtils.decryptOption("base64", encryptUsername);
Review Comment:
do we need to change the identifier to `Base64ConfigShade.getIdentifier()` here,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "Jzjsnow (via GitHub)" <gi...@apache.org>.
Jzjsnow commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1907055452
##########
amoro-ams/src/test/java/org/apache/amoro/server/config/TestConfigShade.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.server.config;
+
+import static org.apache.amoro.server.AmoroServiceContainer.expandConfigMap;
+
+import org.apache.amoro.config.shade.utils.ConfigShadeUtils;
+import org.apache.amoro.server.AmoroManagementConf;
+import org.apache.amoro.shade.guava32.com.google.common.collect.ImmutableMap;
+import org.apache.amoro.shade.guava32.com.google.common.collect.Maps;
+import org.apache.amoro.shade.guava32.com.google.common.io.Resources;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.core.type.TypeReference;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.databind.JsonNode;
+import org.apache.amoro.utils.JacksonUtil;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.yaml.snakeyaml.Yaml;
+
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Map;
+
+public class TestConfigShade {
+ private static final String USERNAME = "admin";
+ private static final String PASSWORD = "password";
+
+ @Test
+ public void testDecryptOptions() {
+ String encryptUsername = "YWRtaW4=";
Review Comment:
In the new commit, a `getBase64EncodedText` method is provided in the unit test to show the base64 encoding process.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3335] Add interface ConfigShade to support encryption of sensitive configuration items and provide a base64 encoding implementation [amoro]
Posted by "Jzjsnow (via GitHub)" <gi...@apache.org>.
Jzjsnow commented on code in PR #3396:
URL: https://github.com/apache/amoro/pull/3396#discussion_r1907055452
##########
amoro-ams/src/test/java/org/apache/amoro/server/config/TestConfigShade.java:
##########
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.server.config;
+
+import static org.apache.amoro.server.AmoroServiceContainer.expandConfigMap;
+
+import org.apache.amoro.config.shade.utils.ConfigShadeUtils;
+import org.apache.amoro.server.AmoroManagementConf;
+import org.apache.amoro.shade.guava32.com.google.common.collect.ImmutableMap;
+import org.apache.amoro.shade.guava32.com.google.common.collect.Maps;
+import org.apache.amoro.shade.guava32.com.google.common.io.Resources;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.core.type.TypeReference;
+import org.apache.amoro.shade.jackson2.com.fasterxml.jackson.databind.JsonNode;
+import org.apache.amoro.utils.JacksonUtil;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.yaml.snakeyaml.Yaml;
+
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.Map;
+
+public class TestConfigShade {
+ private static final String USERNAME = "admin";
+ private static final String PASSWORD = "password";
+
+ @Test
+ public void testDecryptOptions() {
+ String encryptUsername = "YWRtaW4=";
Review Comment:
In the new commit, `TestConfigShade#getBase64EncodedText` is provided to show the base64 encoding process.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org