You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "buu-nguyen (via GitHub)" <gi...@apache.org> on 2025/05/21 04:28:00 UTC
[PR] [Enhance](cdc) Add support for environment variable resolution in parameter mapping [doris-flink-connector]
buu-nguyen opened a new pull request, #597:
URL: https://github.com/apache/doris-flink-connector/pull/597
# Proposed changes
## Problem Summary:
This PR adds support for resolving environment variables within the parameter mapping for CDC tools. It updates the `getConfigMap` method to substitute placeholders like `$VAR` or `${VAR}` with their corresponding environment variable values.
## Checklist(Required)
1. Does it affect the original behavior: No
2. Has unit tests been added: Yes
3. Has document been added or modified: No
4. Does it need to update dependencies: No
5. Are there any changes that cannot be rolled back: No
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhance](cdc) Add support for environment variable resolution in parameter mapping [doris-flink-connector]
Posted by "buu-nguyen (via GitHub)" <gi...@apache.org>.
buu-nguyen closed pull request #597: [Enhance](cdc) Add support for environment variable resolution in parameter mapping
URL: https://github.com/apache/doris-flink-connector/pull/597
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhance](cdc) Add support for environment variable resolution in parameter mapping [doris-flink-connector]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba commented on PR #597:
URL: https://github.com/apache/doris-flink-connector/pull/597#issuecomment-2961376428
ci fails PTAL.@buu-nguyen
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhance](cdc) Add support for environment variable resolution in parameter mapping [doris-flink-connector]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba commented on code in PR #597:
URL: https://github.com/apache/doris-flink-connector/pull/597#discussion_r2108269450
##########
flink-doris-connector/src/main/java/org/apache/doris/flink/tools/cdc/CdcTools.java:
##########
@@ -221,7 +223,33 @@ public static Map<String, String> getConfigMap(MultipleParameterTool params, Str
for (String param : params.getMultiParameter(key)) {
String[] kv = param.split("=", 2);
if (kv.length == 2) {
- map.put(kv[0].trim(), kv[1].trim());
+ String originalValue = kv[1].trim();
+ String resolvedValue = originalValue;
+
+ // Regex to find environment variables like $VAR or ${VAR}
+ Pattern pattern =
+ Pattern.compile(
+ "\\$(?:([A-Za-z_][A-Za-z0-9_]*)|\\{([A-Za-z_][A-Za-z0-9_]*)\\})");
Review Comment:
Can this be put outside the loop?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhance](cdc) Add support for environment variable resolution in parameter mapping [doris-flink-connector]
Posted by "JNSimba (via GitHub)" <gi...@apache.org>.
JNSimba commented on PR #597:
URL: https://github.com/apache/doris-flink-connector/pull/597#issuecomment-2911236690
Thank you for your contribution, can this be added as a case, something like this? https://github.com/apache/doris-flink-connector/blob/master/flink-doris-connector/src/test/resources/container/e2e/mysql2doris/testMySQL2Doris.txt
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhance](cdc) Add support for environment variable resolution in parameter mapping [doris-flink-connector]
Posted by "buu-nguyen (via GitHub)" <gi...@apache.org>.
buu-nguyen commented on code in PR #597:
URL: https://github.com/apache/doris-flink-connector/pull/597#discussion_r2116096861
##########
flink-doris-connector/src/main/java/org/apache/doris/flink/tools/cdc/CdcTools.java:
##########
@@ -221,7 +223,33 @@ public static Map<String, String> getConfigMap(MultipleParameterTool params, Str
for (String param : params.getMultiParameter(key)) {
String[] kv = param.split("=", 2);
if (kv.length == 2) {
- map.put(kv[0].trim(), kv[1].trim());
+ String originalValue = kv[1].trim();
+ String resolvedValue = originalValue;
+
+ // Regex to find environment variables like $VAR or ${VAR}
+ Pattern pattern =
+ Pattern.compile(
+ "\\$(?:([A-Za-z_][A-Za-z0-9_]*)|\\{([A-Za-z_][A-Za-z0-9_]*)\\})");
Review Comment:
Sure, i will make a change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhance](cdc) Add support for environment variable resolution in parameter mapping [doris-flink-connector]
Posted by "buu-nguyen (via GitHub)" <gi...@apache.org>.
buu-nguyen commented on PR #597:
URL: https://github.com/apache/doris-flink-connector/pull/597#issuecomment-2922717222
> Thank you for your contribution, can this be added as a case, something like this? https://github.com/apache/doris-flink-connector/blob/master/flink-doris-connector/src/test/resources/container/e2e/mysql2doris/testMySQL2Doris.txt
Hi @JNSimba, i already added an E2E test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org