You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iggy.apache.org by "tungtose (via GitHub)" <gi...@apache.org> on 2025/12/03 06:56:18 UTC
[PR] feat(connector): add retry mechanism to connectors http config provider [iggy]
tungtose opened a new pull request, #2437:
URL: https://github.com/apache/iggy/pull/2437
This PR addressing #2416
Before:
<img width="1511" height="121" alt="image" src="https://github.com/user-attachments/assets/dc6d211f-10e8-43dc-9f6b-0568a60277ba" />
After:
<img width="1514" height="545" alt="image" src="https://github.com/user-attachments/assets/365f197e-4959-4c3f-a602-23b0c93244dc" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(connectors): add retry mechanism to connectors HTTP config provider [iggy]
Posted by "spetz (via GitHub)" <gi...@apache.org>.
spetz merged PR #2437:
URL: https://github.com/apache/iggy/pull/2437
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(connectors): add retry mechanism to connectors HTTP config provider [iggy]
Posted by "tungtose (via GitHub)" <gi...@apache.org>.
tungtose commented on code in PR #2437:
URL: https://github.com/apache/iggy/pull/2437#discussion_r2584619131
##########
core/connectors/runtime/src/configs/connectors.rs:
##########
@@ -231,6 +231,7 @@ pub async fn create_connectors_config_provider(
) -> Result<Box<dyn ConnectorsConfigProvider>, RuntimeError> {
match config {
RuntimeConnectorsConfig::Local(config) => {
+ tracing::info!("Connector Config: {:?}", config);
Review Comment:
Let me remove this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(connectors): add retry mechanism to connectors HTTP config provider [iggy]
Posted by "tungtose (via GitHub)" <gi...@apache.org>.
tungtose commented on PR #2437:
URL: https://github.com/apache/iggy/pull/2437#issuecomment-3606701363
I found that we may want to add tracing logs to the `sdk` as well. Your call
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(connectors): add retry mechanism to connectors HTTP config provider [iggy]
Posted by "spetz (via GitHub)" <gi...@apache.org>.
spetz commented on PR #2437:
URL: https://github.com/apache/iggy/pull/2437#issuecomment-3606620279
@tungtose looks good! One question, is it possible to somehow log these warnings in a way that it's clear to the user that e.g. fetching the connector config has failed and here's the next retry?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(connectors): add retry mechanism to connectors HTTP config provider [iggy]
Posted by "tungtose (via GitHub)" <gi...@apache.org>.
tungtose commented on code in PR #2437:
URL: https://github.com/apache/iggy/pull/2437#discussion_r2584617959
##########
core/connectors/runtime/config.toml:
##########
@@ -51,3 +51,16 @@ path = "local_state"
[connectors]
config_type = "local"
config_dir = ""
+
+
+### HTTP config type example
Review Comment:
Thanks, I've struggled to find a place to put this config
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(connectors): add retry mechanism to connectors HTTP config provider [iggy]
Posted by "spetz (via GitHub)" <gi...@apache.org>.
spetz commented on PR #2437:
URL: https://github.com/apache/iggy/pull/2437#issuecomment-3606873025
> I found that we may want to add tracing logs to the `sdk` as well. Your call
Ok, let's do this next, depending on how many changes will be needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(connectors): add retry mechanism to connectors HTTP config provider [iggy]
Posted by "mmodzelewski (via GitHub)" <gi...@apache.org>.
mmodzelewski commented on code in PR #2437:
URL: https://github.com/apache/iggy/pull/2437#discussion_r2584569780
##########
core/connectors/runtime/config.toml:
##########
@@ -51,3 +51,16 @@ path = "local_state"
[connectors]
config_type = "local"
config_dir = ""
+
+
+### HTTP config type example
Review Comment:
please move this part into runtime's readme file, there's an http provider section where you can include this
##########
core/connectors/runtime/src/configs/connectors.rs:
##########
@@ -231,6 +231,7 @@ pub async fn create_connectors_config_provider(
) -> Result<Box<dyn ConnectorsConfigProvider>, RuntimeError> {
match config {
RuntimeConnectorsConfig::Local(config) => {
+ tracing::info!("Connector Config: {:?}", config);
Review Comment:
Maybe use a debug level here. The config dir is already logged during provider init
##########
core/integration/tests/connectors/http_config_provider/config_direct.toml:
##########
@@ -18,3 +18,10 @@
[connectors]
config_type = "http"
base_url = "http://localhost:8080"
+
+[connectors.retry]
Review Comment:
as this is already default, I think you can skip it from here and `config_wrapped.toml`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(connectors): add retry mechanism to connectors HTTP config provider [iggy]
Posted by "tungtose (via GitHub)" <gi...@apache.org>.
tungtose commented on PR #2437:
URL: https://github.com/apache/iggy/pull/2437#issuecomment-3606380258
Thanks @mmodzelewski. I've resolved your comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org