You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@xtable.apache.org by "the-other-tim-brown (via GitHub)" <gi...@apache.org> on 2024/12/08 20:52:26 UTC
[PR] add a loop for continuous conversion with the RunSync tool [incubator-xtable]
the-other-tim-brown opened a new pull request, #593:
URL: https://github.com/apache/incubator-xtable/pull/593
## *Important Read*
- *Please ensure the GitHub issue is mentioned at the beginning of the PR*
## What is the purpose of the pull request
*(For example: This pull request implements the sync for delta format.)*
## Brief change log
*(for example:)*
- *Fixed JSON parsing error when persisting state*
- *Added unit tests for schema evolution*
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added TestConversionController to verify the change.*
- *Manually verified the change by running a job locally.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [594] Support continuous conversion with the RunSync tool [incubator-xtable]
Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #593:
URL: https://github.com/apache/incubator-xtable/pull/593#discussion_r1987778126
##########
xtable-utilities/src/main/java/org/apache/xtable/utilities/RunSync.java:
##########
@@ -115,11 +132,39 @@ public static void main(String[] args) throws IOException {
return;
}
- DatasetConfig datasetConfig = new DatasetConfig();
+ if (cmd.hasOption(CONTINUOUS_MODE)) {
+ ScheduledExecutorService executorService = Executors.newScheduledThreadPool(1);
+ long intervalInSeconds = Long.parseLong(cmd.getOptionValue(CONTINUOUS_MODE_INTERVAL, "5"));
+ executorService.scheduleAtFixedRate(
+ () -> {
+ try {
+ runSync(cmd);
+ } catch (IOException ex) {
+ log.error("Sync operation failed", ex);
+ }
+ },
+ 0,
+ intervalInSeconds,
+ TimeUnit.SECONDS);
+ while (!Thread.interrupted()) {
Review Comment:
nice catch, I need to check the current thread which has an instance method which does not clear the state
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [594] Support continuous conversion with the RunSync tool [incubator-xtable]
Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown merged PR #593:
URL: https://github.com/apache/incubator-xtable/pull/593
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [594] Support continuous conversion with the RunSync tool [incubator-xtable]
Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 commented on code in PR #593:
URL: https://github.com/apache/incubator-xtable/pull/593#discussion_r1987772434
##########
xtable-utilities/src/main/java/org/apache/xtable/utilities/RunSync.java:
##########
@@ -115,11 +132,39 @@ public static void main(String[] args) throws IOException {
return;
}
- DatasetConfig datasetConfig = new DatasetConfig();
+ if (cmd.hasOption(CONTINUOUS_MODE)) {
+ ScheduledExecutorService executorService = Executors.newScheduledThreadPool(1);
+ long intervalInSeconds = Long.parseLong(cmd.getOptionValue(CONTINUOUS_MODE_INTERVAL, "5"));
+ executorService.scheduleAtFixedRate(
+ () -> {
+ try {
+ runSync(cmd);
+ } catch (IOException ex) {
+ log.error("Sync operation failed", ex);
+ }
+ },
+ 0,
+ intervalInSeconds,
+ TimeUnit.SECONDS);
+ while (!Thread.interrupted()) {
Review Comment:
This method clears the interrupted status for the thread, do we need that here ?
`Thread.currentThread().isInterrupted(false)` doesn't clear the interrupted state after the check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [594] Support continuous conversion with the RunSync tool [incubator-xtable]
Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 commented on PR #593:
URL: https://github.com/apache/incubator-xtable/pull/593#issuecomment-2711437382
Can you squash into one commit before merging ? Thanks !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org