You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/04/11 06:14:11 UTC
[GitHub] [beam] Akshay-Iyangar opened a new pull request #11396: [BEAM-9742]
Add Configurable FluentBackoff to JdbcIO Write
Akshay-Iyangar opened a new pull request #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write
URL: https://github.com/apache/beam/pull/11396
Add Configurable FluentBackoff to JdbcIO Write
[Jira ticket](https://issues.apache.org/jira/browse/BEAM-9742)
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lukecwik commented on a change in pull request #11396:
[BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write
URL: https://github.com/apache/beam/pull/11396#discussion_r409007908
##########
File path: sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java
##########
@@ -257,6 +258,28 @@ public boolean apply(SQLException e) {
}
}
+ /**
+ * This is the default {@link FluentBackoffConfiguration} that we use to retry when a {@link
+ * SQLException} occurs.
+ */
+ public static class DefaultFluentBackoffConfiguration implements FluentBackoffConfiguration {
Review comment:
We can't make FluentBackoff part of the public API surface since it is in the `util` package. It looks like other IO connectors have been creating their own `RetryConfiguration` class such as:
SnsIO: https://github.com/apache/beam/blob/da9e17288e8473925674a4691d9e86252e67d7d7/sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/sns/SnsIO.java#L262
SolrIO: https://github.com/apache/beam/blob/da9e17288e8473925674a4691d9e86252e67d7d7/sdks/java/io/solr/src/main/java/org/apache/beam/sdk/io/solr/SolrIO.java#L225
and then converting them to any internal implementation that makes sense.
Making FluentBackoff or a RetryConfiguration public and shared across implementation might make sense but warrants a discussion on the dev@ mailing list to see what the community thinks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lukecwik commented on a change in pull request #11396:
[BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write
Posted by GitBox <gi...@apache.org>.
lukecwik commented on a change in pull request #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write
URL: https://github.com/apache/beam/pull/11396#discussion_r409843690
##########
File path: sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java
##########
@@ -257,6 +258,28 @@ public boolean apply(SQLException e) {
}
}
+ /**
+ * This is the default {@link FluentBackoffConfiguration} that we use to retry when a {@link
+ * SQLException} occurs.
+ */
+ public static class DefaultFluentBackoffConfiguration implements FluentBackoffConfiguration {
Review comment:
Waiting on updates on ML thread:
https://lists.apache.org/thread.html/r7fde7b5c87c6008689a013fc113d869d3ac15cbc7a35e4534469b9ab%40%3Cdev.beam.apache.org%3E
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] Akshay-Iyangar commented on a change in pull request #11396:
[BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write
Posted by GitBox <gi...@apache.org>.
Akshay-Iyangar commented on a change in pull request #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write
URL: https://github.com/apache/beam/pull/11396#discussion_r409062570
##########
File path: sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java
##########
@@ -257,6 +258,28 @@ public boolean apply(SQLException e) {
}
}
+ /**
+ * This is the default {@link FluentBackoffConfiguration} that we use to retry when a {@link
+ * SQLException} occurs.
+ */
+ public static class DefaultFluentBackoffConfiguration implements FluentBackoffConfiguration {
Review comment:
@lukecwik - cool! I will mail the dev group.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] Akshay-Iyangar commented on issue #11396: [BEAM-9742] Add
Configurable FluentBackoff to JdbcIO Write
Posted by GitBox <gi...@apache.org>.
Akshay-Iyangar commented on issue #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write
URL: https://github.com/apache/beam/pull/11396#issuecomment-612341803
@aromanenko-dev @iemejia @lukecwik
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services