You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/06/20 10:16:53 UTC
[GitHub] [hudi] LinMingQiang opened a new pull request, #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
LinMingQiang opened a new pull request, #5884:
URL: https://github.com/apache/hudi/pull/5884
…ystemview', and support the configuration of request time out and retry times
Signed-off-by: LinMingQiang <13...@qq.com>
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
https://issues.apache.org/jira/browse/HUDI-3669
https://github.com/apache/hudi/issues/5832
*(For example: This pull request adds quick-start document.)*
## Brief change log
*(for example:)*
- *Modify AnnotationLocation checkstyle rule in checkstyle.xml*
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added HoodieClientWriteTest to verify the change.*
- *Manually verified the change by running a job locally.*
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r900669178
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/util/ViewStorageProperties.java:
##########
@@ -73,6 +73,8 @@ public static FileSystemViewStorageConfig loadFromProperties(String basePath, Co
try (FSDataInputStream inputStream = fs.open(propertyPath)) {
props.load(inputStream);
}
+ // reset filesystem view storage properties from user conf
+ props.putAll(conf.toMap());
Review Comment:
What does this sentence mean? ：`You should fix the rebuilt path, something like StreamerUtil: line 427 for master.`
Do I need to check this `propertyPath`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1158704703
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f4f87fb17455c85bb4241215af93195946c3f8c1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354)
* fe9bb87af45b46a242fdc253e3d6338e03a345d8 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang closed pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang closed pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
URL: https://github.com/apache/hudi/pull/5884
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1171921926
@danny0405 , can you help to review this PR if you have time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1183053118
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r901339933
##########
hudi-common/src/main/java/org/apache/hudi/common/util/RetryHelper.java:
##########
@@ -18,19 +18,21 @@
package org.apache.hudi.common.util;
+import org.apache.hudi.exception.HoodieException;
import org.apache.log4j.LogManager;
import org.apache.log4j.Logger;
import java.io.IOException;
+import java.io.Serializable;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Random;
import java.util.stream.Collectors;
-public class RetryHelper<T> {
+public class RetryHelper<T> implements Serializable {
private static final Logger LOG = LogManager.getLogger(RetryHelper.class);
- private CheckedFunction<T> func;
+ private transient CheckedFunction<T> func;
private int num;
private long maxIntervalTime;
private long initialIntervalTime = 100L;
Review Comment:
Can we make these variables final ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1206739122
@LinMingQiang : LGTM. thanks for the contribution. I assume, you have tested this piece of code in your env (staging/production) and its all good. can you confirm please.
is it possible to write tests to cover the new code written ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207219976
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
}, {
"hash" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a3c7abdbd0564a4e32ec797870d0571e7576f7b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628)
* 73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r955568822
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -165,17 +179,9 @@ private <T> T executeRequest(String requestPath, Map<String, String> queryParame
String url = builder.toString();
LOG.info("Sending request : (" + url + ")");
- Response response;
- int timeout = this.timeoutSecs * 1000; // msec
- switch (method) {
- case GET:
- response = Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- case POST:
- default:
- response = Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- }
+ // Reset url and method, to avoid repeatedly instantiating objects.
+ urlCheckedFunc.setUrlAndMethod(url, method);
Review Comment:
> we should have been more careful here
+10086, we may need more attention for the core part code since there are many users now, the core change should be conservative.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207220768
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
}, {
"hash" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640",
"triggerID" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a3c7abdbd0564a4e32ec797870d0571e7576f7b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628)
* 73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] didiaode18 commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
didiaode18 commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207321318
I had the same problem, since I applied 3669.patch.zip, my production cluster had no errors
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r900100002
##########
hudi-common/src/main/java/org/apache/hudi/common/util/RetryHelper.java:
##########
@@ -80,7 +92,7 @@ public T start() throws IOException {
LOG.error("Still failed to " + taskInfo + " after retried " + num + " times.", e);
throw e;
}
- LOG.warn("Catch Exception " + taskInfo + ", will retry after " + waitTime + " ms.", e);
+ LOG.warn("Catch Exception When " + taskInfo + ", will retry after " + waitTime + " ms.", e);
try {
Review Comment:
When -> for
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1160241094
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4f7255e16bb3ee68213d9e4b8632d6d33178a352 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364)
* 714205fdaf3700e93c94075ab73f9f54c354db2b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395)
* 1d8137c96b0260847dbe6ffc07fe28f46bc28168 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1159352044
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4f7255e16bb3ee68213d9e4b8632d6d33178a352 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1158765761
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fe9bb87af45b46a242fdc253e3d6338e03a345d8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1159343713
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362)
* 4f7255e16bb3ee68213d9e4b8632d6d33178a352 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207228859
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
}, {
"hash" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640",
"triggerID" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r939581543
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -495,4 +501,33 @@ public Option<HoodieBaseFile> getLatestBaseFile(String partitionPath, String fil
throw new HoodieRemoteException(e);
}
}
+
+ /**
+ * For remote HTTP requests, to avoid repeatedly instantiating objects.
+ */
+ private class HttpRequestCheckedFunction implements RetryHelper.CheckedFunction<Response> {
+ private String url;
+ private RequestMethod method;
+ private final int timeout;
Review Comment:
can you suffix the unit to the variable. timeoutMs or timeoutSecs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207324197
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
}, {
"hash" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640",
"triggerID" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"triggerType" : "PUSH"
}, {
"hash" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10641",
"triggerID" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"triggerType" : "PUSH"
}, {
"hash" : "7598979abfad4d436f6f8fbe10428b6bc8f46203",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10646",
"triggerID" : "7598979abfad4d436f6f8fbe10428b6bc8f46203",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7598979abfad4d436f6f8fbe10428b6bc8f46203 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10646)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1171924522
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8cbe40dfa825fbf8dca54cf7fda4b0893441e14e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584)
* 8c7cebb21fbeebddf99e857fed394680dddc9019 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r900669178
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/util/ViewStorageProperties.java:
##########
@@ -73,6 +73,8 @@ public static FileSystemViewStorageConfig loadFromProperties(String basePath, Co
try (FSDataInputStream inputStream = fs.open(propertyPath)) {
props.load(inputStream);
}
+ // reset filesystem view storage properties from user conf
+ props.putAll(conf.toMap());
Review Comment:
What do you mean? Do I need to check this `propertyPath`？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207229695
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
}, {
"hash" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640",
"triggerID" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"triggerType" : "PUSH"
}, {
"hash" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640)
* e2887ca8c35944f6a9fde26b78013ce360c9cf2b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1203404419
@nsivabalan can you help to review this PR if you have time,This PR has helped many people solve the problem,So I think it can be added to 0.12.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1160170361
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4f7255e16bb3ee68213d9e4b8632d6d33178a352 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364)
* 714205fdaf3700e93c94075ab73f9f54c354db2b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1167028410
> Thanks for the contribution, i have reviewed and apply a patch: [3669.patch.zip](https://github.com/apache/hudi/files/8989811/3669.patch.zip)
Thank you very much for your valuable advice, I have finished merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r955363946
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -165,17 +179,9 @@ private <T> T executeRequest(String requestPath, Map<String, String> queryParame
String url = builder.toString();
LOG.info("Sending request : (" + url + ")");
- Response response;
- int timeout = this.timeoutSecs * 1000; // msec
- switch (method) {
- case GET:
- response = Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- case POST:
- default:
- response = Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- }
+ // Reset url and method, to avoid repeatedly instantiating objects.
+ urlCheckedFunc.setUrlAndMethod(url, method);
Review Comment:
this is not thread-safe actually. Might be impacting even regular flow even if retries are not enabled :( we should have been more careful here. urlCheckedFunc is an instance variable accessed by multiple threads concurrently could result in expected responses.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207249253
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
}, {
"hash" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640",
"triggerID" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"triggerType" : "PUSH"
}, {
"hash" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10641",
"triggerID" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e2887ca8c35944f6a9fde26b78013ce360c9cf2b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10641)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207137825
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8c7cebb21fbeebddf99e857fed394680dddc9019 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664)
* 5a3c7abdbd0564a4e32ec797870d0571e7576f7b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207231253
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
}, {
"hash" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640",
"triggerID" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"triggerType" : "PUSH"
}, {
"hash" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10641",
"triggerID" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640)
* e2887ca8c35944f6a9fde26b78013ce360c9cf2b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10641)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1158595640
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 16c1d84cf0a295319aeb97df653fe35d06e2eb31 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335)
* f4f87fb17455c85bb4241215af93195946c3f8c1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r899044165
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -165,16 +167,29 @@ private <T> T executeRequest(String requestPath, Map<String, String> queryParame
String url = builder.toString();
LOG.info("Sending request : (" + url + ")");
- Response response;
+ Response response = null;
int timeout = this.timeoutSecs * 1000; // msec
- switch (method) {
- case GET:
- response = Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- case POST:
- default:
- response = Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
+ int retryCounts = 0;
+ while (retryCounts <= maxRetries) {
+ try {
+ switch (method) {
Review Comment:
I intend to rewrite the construction method of `Remotehoodietablefilesystemview`, like `Remotehoodietablefilesystemview (Hoodietablemetaclient metaclient, Filesystemviewstorageconfig viewconf)`, which can support more configurations and facilitate expansion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1167055341
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1d8137c96b0260847dbe6ffc07fe28f46bc28168 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397)
* 8cbe40dfa825fbf8dca54cf7fda4b0893441e14e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1167049923
Hi, @nsivabalan , can you help to review this PR if you have time, i have left a minor comment for you,
and it is great if you have some suggestion for testing the fs view in retry case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r910779768
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -165,17 +176,17 @@ private <T> T executeRequest(String requestPath, Map<String, String> queryParame
String url = builder.toString();
LOG.info("Sending request : (" + url + ")");
- Response response;
int timeout = this.timeoutSecs * 1000; // msec
- switch (method) {
- case GET:
- response = Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- case POST:
- default:
- response = Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- }
+ RetryHelper.CheckedFunction<Response> checkedFun = () -> {
+ switch (method) {
+ case GET:
+ return Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
+ case POST:
+ default:
+ return Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
Review Comment:
Hi, @nsivabalan , can you help to review this PR if you have time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1171926608
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8cbe40dfa825fbf8dca54cf7fda4b0893441e14e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584)
* 8c7cebb21fbeebddf99e857fed394680dddc9019 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1183054828
@danny0405 , can you help to review this PR if you have time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1157349674
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 16c1d84cf0a295319aeb97df653fe35d06e2eb31 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1158701145
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f4f87fb17455c85bb4241215af93195946c3f8c1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207108160
Yes, I'm sure. I encountered the problem of request failure in the production environment, and used this PR to solve and run stably. I will try to finish the test today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207137293
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8c7cebb21fbeebddf99e857fed394680dddc9019 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664)
* 5a3c7abdbd0564a4e32ec797870d0571e7576f7b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1158898185
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1160165545
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4f7255e16bb3ee68213d9e4b8632d6d33178a352 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364)
* 714205fdaf3700e93c94075ab73f9f54c354db2b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r901448556
##########
hudi-common/src/main/java/org/apache/hudi/common/util/RetryHelper.java:
##########
@@ -79,30 +72,35 @@ public T start() throws IOException {
int retries = 0;
T functionResult = null;
- while (true) {
- long waitTime = Math.min(getWaitTimeExp(retries), maxIntervalTime);
- try {
- functionResult = func.get();
- break;
- } catch (IOException | RuntimeException e) {
- if (!checkIfExceptionInRetryList(e)) {
- throw e;
- }
- if (retries++ >= num) {
- LOG.error("Still failed to " + taskInfo + " after retried " + num + " times.", e);
- throw e;
- }
- LOG.warn("Catch Exception for " + taskInfo + ", will retry after " + waitTime + " ms.", e);
+ // Disable retry.
+ if (num <= 0) {
+ functionResult = func.get();
Review Comment:
If retry is disabled, we should use the `RetryHelper` anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1160245808
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4f7255e16bb3ee68213d9e4b8632d6d33178a352 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364)
* 714205fdaf3700e93c94075ab73f9f54c354db2b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395)
* 1d8137c96b0260847dbe6ffc07fe28f46bc28168 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r907127350
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -165,17 +176,17 @@ private <T> T executeRequest(String requestPath, Map<String, String> queryParame
String url = builder.toString();
LOG.info("Sending request : (" + url + ")");
- Response response;
int timeout = this.timeoutSecs * 1000; // msec
- switch (method) {
- case GET:
- response = Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- case POST:
- default:
- response = Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- }
+ RetryHelper.CheckedFunction<Response> checkedFun = () -> {
+ switch (method) {
+ case GET:
+ return Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
+ case POST:
+ default:
+ return Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
Review Comment:
Can we add a parameter `url` to `RetryHelper.CheckedFunction`, In this way, each time call only the `url` attribute of `CheckedFunction` needs to be modified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1167395342
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8cbe40dfa825fbf8dca54cf7fda4b0893441e14e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1157585954
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 16c1d84cf0a295319aeb97df653fe35d06e2eb31 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang closed pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang closed pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
URL: https://github.com/apache/hudi/pull/5884
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207224617
@hudi-bot run azure re-run the last Azure build
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1183053294
> ## CI report:
> * [8c7cebb](https://github.com/apache/hudi/commit/8c7cebb21fbeebddf99e857fed394680dddc9019) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664)
>
> Bot commands
> @hudi-bot supports the following commands:
> * `@hudi-bot run azure` re-run the last Azure build
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207309776
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
}, {
"hash" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640",
"triggerID" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"triggerType" : "PUSH"
}, {
"hash" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10641",
"triggerID" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"triggerType" : "PUSH"
}, {
"hash" : "7598979abfad4d436f6f8fbe10428b6bc8f46203",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10646",
"triggerID" : "7598979abfad4d436f6f8fbe10428b6bc8f46203",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e2887ca8c35944f6a9fde26b78013ce360c9cf2b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10641)
* 7598979abfad4d436f6f8fbe10428b6bc8f46203 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10646)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r900097971
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/FileSystemViewStorageConfig.java:
##########
@@ -110,6 +110,30 @@ public class FileSystemViewStorageConfig extends HoodieConfig {
.defaultValue(5 * 60) // 5 min
.withDocumentation("Timeout in seconds, to wait for API requests against a remote file system view. e.g timeline server.");
+ public static final ConfigProperty<Integer> REMOTE_MAX_RETRY_NUMBERS = ConfigProperty
+ .key("hoodie.filesystem.view.remote.retry.max_numbers")
+ .defaultValue(3) // 3 times
+ .withDocumentation("Maximum number of retry for API requests against a remote file system view. e.g timeline server.");
+
+ public static final ConfigProperty<Long> REMOTE_INITIAL_RETRY_INTERVAL_MS = ConfigProperty
+ .key("hoodie.filesystem.view.remote.retry.initial_interval_ms")
+ .defaultValue(100L)
+ .sinceVersion("0.11.0")
+ .withDocumentation("Amount of time (in ms) to wait, before retry to do operations on storage.");
+
+ public static final ConfigProperty<Long> REMOTE_MAX_RETRY_INTERVAL_MS = ConfigProperty
+ .key("hoodie.filesystem.view.remote.retry.max_interval_ms")
+ .defaultValue(2000L)
+ .sinceVersion("0.11.0")
+ .withDocumentation("Maximum amount of time (in ms), to wait for next retry.");
+
+ public static final ConfigProperty<String> RETRY_EXCEPTIONS = ConfigProperty
+ .key("hoodie.filesystem.view.remote.retry.exceptions")
+ .defaultValue("")
+ .sinceVersion("0.11.0")
Review Comment:
since 0.12.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r900103569
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/util/ViewStorageProperties.java:
##########
@@ -73,6 +73,8 @@ public static FileSystemViewStorageConfig loadFromProperties(String basePath, Co
try (FSDataInputStream inputStream = fs.open(propertyPath)) {
props.load(inputStream);
}
+ // reset filesystem view storage properties from user conf
+ props.putAll(conf.toMap());
Review Comment:
Shouldn't the dump already handle all the options ?
You should fix the rebuilt path, something like StreamerUtil: line 427 for master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r899017812
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -165,16 +167,29 @@ private <T> T executeRequest(String requestPath, Map<String, String> queryParame
String url = builder.toString();
LOG.info("Sending request : (" + url + ")");
- Response response;
+ Response response = null;
int timeout = this.timeoutSecs * 1000; // msec
- switch (method) {
- case GET:
- response = Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- case POST:
- default:
- response = Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
+ int retryCounts = 0;
+ while (retryCounts <= maxRetries) {
+ try {
+ switch (method) {
Review Comment:
Do we also need to add `retry.max_ interval_ ms`,`retry.initial_ interval_ ms` and `retry.exceptions` parameter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r898888250
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -137,17 +138,18 @@ private enum RequestMethod {
}
public RemoteHoodieTableFileSystemView(String server, int port, HoodieTableMetaClient metaClient) {
- this(server, port, metaClient, 300);
+ this(server, port, metaClient, 300, 3);
}
Review Comment:
A better way is to fetch the constant from the config option.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -165,16 +167,29 @@ private <T> T executeRequest(String requestPath, Map<String, String> queryParame
String url = builder.toString();
LOG.info("Sending request : (" + url + ")");
- Response response;
+ Response response = null;
int timeout = this.timeoutSecs * 1000; // msec
- switch (method) {
- case GET:
- response = Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- case POST:
- default:
- response = Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
+ int retryCounts = 0;
+ while (retryCounts <= maxRetries) {
+ try {
+ switch (method) {
Review Comment:
We introduced a retry helper in https://github.com/apache/hudi/pull/3887, with richer strategies, maybe you can take a reference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1157353184
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 16c1d84cf0a295319aeb97df653fe35d06e2eb31 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1183060336
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 8c7cebb21fbeebddf99e857fed394680dddc9019 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1167019018
Thanks for the contribution, i have reviewed and apply a patch:
[3669.patch.zip](https://github.com/apache/hudi/files/8989811/3669.patch.zip)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1167050536
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1d8137c96b0260847dbe6ffc07fe28f46bc28168 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397)
* 8cbe40dfa825fbf8dca54cf7fda4b0893441e14e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1171983624
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8c7cebb21fbeebddf99e857fed394680dddc9019 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207144973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a3c7abdbd0564a4e32ec797870d0571e7576f7b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207324615
thanks @LinMingQiang @didiaode18 for confirming.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
nsivabalan merged PR #5884:
URL: https://github.com/apache/hudi/pull/5884
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207403146
thanks @didiaode18 for confirming.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r955364712
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -165,17 +179,9 @@ private <T> T executeRequest(String requestPath, Map<String, String> queryParame
String url = builder.toString();
LOG.info("Sending request : (" + url + ")");
- Response response;
- int timeout = this.timeoutSecs * 1000; // msec
- switch (method) {
- case GET:
- response = Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- case POST:
- default:
- response = Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- }
+ // Reset url and method, to avoid repeatedly instantiating objects.
+ urlCheckedFunc.setUrlAndMethod(url, method);
+ Response response = retryHelper != null ? retryHelper.tryWith(urlCheckedFunc).start() : urlCheckedFunc.get();
Review Comment:
@LinMingQiang @danny0405 Every request goes through this flow and the `urlCheckedFunc` should not be shared across requests. The logic here is incorrect. This can cause serious correctness problems under concurrency.
We need to revert this logic. Also suggest that we should guard against such changes in the hot path with a flag.
cc @nsivabalan @rmahindra123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r900100746
##########
hudi-common/src/main/java/org/apache/hudi/common/util/RetryHelper.java:
##########
@@ -47,13 +48,24 @@ public RetryHelper(long maxRetryIntervalMs, int maxRetryNumbers, long initialRet
if (StringUtils.isNullOrEmpty(retryExceptions)) {
this.retryExceptionsClasses = new ArrayList<>();
} else {
- this.retryExceptionsClasses = Arrays.stream(retryExceptions.split(","))
- .map(exception -> (Exception) ReflectionUtils.loadClass(exception, ""))
- .map(Exception::getClass)
- .collect(Collectors.toList());
+ try {
+ this.retryExceptionsClasses = Arrays.stream(retryExceptions.split(","))
+ .map(exception -> (Exception) ReflectionUtils.loadClass(exception, ""))
+ .map(Exception::getClass)
+ .collect(Collectors.toList());
+ } catch (HoodieException e) {
Review Comment:
Why this change ? If there is no default clazz for retryExceptions, should skip this construction ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r900103569
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/util/ViewStorageProperties.java:
##########
@@ -73,6 +73,8 @@ public static FileSystemViewStorageConfig loadFromProperties(String basePath, Co
try (FSDataInputStream inputStream = fs.open(propertyPath)) {
props.load(inputStream);
}
+ // reset filesystem view storage properties from user conf
+ props.putAll(conf.toMap());
Review Comment:
Shouldn't the dump already handle all the options ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r900118999
##########
hudi-common/src/main/java/org/apache/hudi/common/util/RetryHelper.java:
##########
@@ -47,13 +48,24 @@ public RetryHelper(long maxRetryIntervalMs, int maxRetryNumbers, long initialRet
if (StringUtils.isNullOrEmpty(retryExceptions)) {
this.retryExceptionsClasses = new ArrayList<>();
} else {
- this.retryExceptionsClasses = Arrays.stream(retryExceptions.split(","))
- .map(exception -> (Exception) ReflectionUtils.loadClass(exception, ""))
- .map(Exception::getClass)
- .collect(Collectors.toList());
+ try {
+ this.retryExceptionsClasses = Arrays.stream(retryExceptions.split(","))
+ .map(exception -> (Exception) ReflectionUtils.loadClass(exception, ""))
+ .map(Exception::getClass)
+ .collect(Collectors.toList());
+ } catch (HoodieException e) {
Review Comment:
If the filled `Exception class name` does not exist,`java.lang.ClassNotFoundException` a exception will be thrown.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r900096742
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -165,16 +167,29 @@ private <T> T executeRequest(String requestPath, Map<String, String> queryParame
String url = builder.toString();
LOG.info("Sending request : (" + url + ")");
- Response response;
+ Response response = null;
int timeout = this.timeoutSecs * 1000; // msec
- switch (method) {
- case GET:
- response = Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- case POST:
- default:
- response = Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
+ int retryCounts = 0;
+ while (retryCounts <= maxRetries) {
+ try {
+ switch (method) {
Review Comment:
That's a good idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r900669178
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/util/ViewStorageProperties.java:
##########
@@ -73,6 +73,8 @@ public static FileSystemViewStorageConfig loadFromProperties(String basePath, Co
try (FSDataInputStream inputStream = fs.open(propertyPath)) {
props.load(inputStream);
}
+ // reset filesystem view storage properties from user conf
+ props.putAll(conf.toMap());
Review Comment:
What does this sentence mean? ：`You should fix the rebuilt path, something like StreamerUtil: line 427 for master.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1160415156
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 714205fdaf3700e93c94075ab73f9f54c354db2b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395)
* 1d8137c96b0260847dbe6ffc07fe28f46bc28168 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1207309277
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
}, {
"hash" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10628",
"triggerID" : "5a3c7abdbd0564a4e32ec797870d0571e7576f7b",
"triggerType" : "PUSH"
}, {
"hash" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10640",
"triggerID" : "73f8b8cfca5a6ae9b52ac57d37e4cdf7415dc444",
"triggerType" : "PUSH"
}, {
"hash" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10641",
"triggerID" : "e2887ca8c35944f6a9fde26b78013ce360c9cf2b",
"triggerType" : "PUSH"
}, {
"hash" : "7598979abfad4d436f6f8fbe10428b6bc8f46203",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7598979abfad4d436f6f8fbe10428b6bc8f46203",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e2887ca8c35944f6a9fde26b78013ce360c9cf2b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10641)
* 7598979abfad4d436f6f8fbe10428b6bc8f46203 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r939591061
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -495,4 +501,33 @@ public Option<HoodieBaseFile> getLatestBaseFile(String partitionPath, String fil
throw new HoodieRemoteException(e);
}
}
+
+ /**
+ * For remote HTTP requests, to avoid repeatedly instantiating objects.
+ */
+ private class HttpRequestCheckedFunction implements RetryHelper.CheckedFunction<Response> {
+ private String url;
+ private RequestMethod method;
+ private final int timeout;
Review Comment:
OK, modified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang closed pull request #5884: [HUDI-3669] Add a request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
LinMingQiang closed pull request #5884: [HUDI-3669] Add a request retry mechanism for 'Remotehoodietablefiles…
URL: https://github.com/apache/hudi/pull/5884
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1158708094
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f4f87fb17455c85bb4241215af93195946c3f8c1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354)
* fe9bb87af45b46a242fdc253e3d6338e03a345d8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1158859803
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fe9bb87af45b46a242fdc253e3d6338e03a345d8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358)
* ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1159344574
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362)
* 4f7255e16bb3ee68213d9e4b8632d6d33178a352 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r901346787
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -165,17 +174,16 @@ private <T> T executeRequest(String requestPath, Map<String, String> queryParame
String url = builder.toString();
LOG.info("Sending request : (" + url + ")");
- Response response;
int timeout = this.timeoutSecs * 1000; // msec
- switch (method) {
- case GET:
- response = Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- case POST:
- default:
- response = Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- }
+ Response response = (Response) retryHelper.tryWith(() -> {
+ switch (method) {
+ case GET:
+ return Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
+ case POST:
+ default:
+ return Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
+ }
+ }).start();
Review Comment:
Can we by default disable this feature ? You can add a new option similar like `hoodie.filesystem.operation.retry.enable`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1158599111
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 16c1d84cf0a295319aeb97df653fe35d06e2eb31 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335)
* f4f87fb17455c85bb4241215af93195946c3f8c1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1183056392
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
}, {
"hash" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9584",
"triggerID" : "8cbe40dfa825fbf8dca54cf7fda4b0893441e14e",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664",
"triggerID" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"triggerType" : "PUSH"
}, {
"hash" : "8c7cebb21fbeebddf99e857fed394680dddc9019",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1183053118",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 8c7cebb21fbeebddf99e857fed394680dddc9019 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9664)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5884:
URL: https://github.com/apache/hudi/pull/5884#issuecomment-1160486162
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9335",
"triggerID" : "16c1d84cf0a295319aeb97df653fe35d06e2eb31",
"triggerType" : "PUSH"
}, {
"hash" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9354",
"triggerID" : "f4f87fb17455c85bb4241215af93195946c3f8c1",
"triggerType" : "PUSH"
}, {
"hash" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9358",
"triggerID" : "fe9bb87af45b46a242fdc253e3d6338e03a345d8",
"triggerType" : "PUSH"
}, {
"hash" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9362",
"triggerID" : "ba37d53cdb11f290a09a4d7d3926fd3ac4ba8632",
"triggerType" : "PUSH"
}, {
"hash" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9364",
"triggerID" : "4f7255e16bb3ee68213d9e4b8632d6d33178a352",
"triggerType" : "PUSH"
}, {
"hash" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9395",
"triggerID" : "714205fdaf3700e93c94075ab73f9f54c354db2b",
"triggerType" : "PUSH"
}, {
"hash" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397",
"triggerID" : "1d8137c96b0260847dbe6ffc07fe28f46bc28168",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1d8137c96b0260847dbe6ffc07fe28f46bc28168 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9397)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5884: [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5884:
URL: https://github.com/apache/hudi/pull/5884#discussion_r907118343
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -165,17 +176,17 @@ private <T> T executeRequest(String requestPath, Map<String, String> queryParame
String url = builder.toString();
LOG.info("Sending request : (" + url + ")");
- Response response;
int timeout = this.timeoutSecs * 1000; // msec
- switch (method) {
- case GET:
- response = Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- case POST:
- default:
- response = Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
- break;
- }
+ RetryHelper.CheckedFunction<Response> checkedFun = () -> {
+ switch (method) {
+ case GET:
+ return Request.Get(url).connectTimeout(timeout).socketTimeout(timeout).execute();
+ case POST:
+ default:
+ return Request.Post(url).connectTimeout(timeout).socketTimeout(timeout).execute();
Review Comment:
@nsivabalan , can you help with this part if you have time, the code is not efficient compared to the original, wondering if you have some good idea here ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org