You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "damondouglas (via GitHub)" <gi...@apache.org> on 2023/09/01 23:01:11 UTC
[GitHub] [beam] damondouglas opened a new pull request, #28288: Remove warning from catch in table exists validation
damondouglas opened a new pull request, #28288:
URL: https://github.com/apache/beam/pull/28288
This PR addresses #28080 by refactoring the catch block during BigtableIO table existence validation.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [x] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
~Update `CHANGES.md` with noteworthy changes.~
~If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).~
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on pull request #28288: Remove warning from catch in table exists validation
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #28288:
URL: https://github.com/apache/beam/pull/28288#issuecomment-1708514618
from the existing warning message `"proceeding"` this sounds intended. throw an exception may introduce breaking change R: @mutianf
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damondouglas merged pull request #28288: Remove warning from catch in table exists validation
Posted by "damondouglas (via GitHub)" <gi...@apache.org>.
damondouglas merged PR #28288:
URL: https://github.com/apache/beam/pull/28288
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mutianf commented on a diff in pull request #28288: Remove warning from catch in table exists validation
Posted by "mutianf (via GitHub)" <gi...@apache.org>.
mutianf commented on code in PR #28288:
URL: https://github.com/apache/beam/pull/28288#discussion_r1330166796
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java:
##########
@@ -688,14 +688,13 @@ public final String toString() {
private void validateTableExists(
BigtableConfig config, BigtableReadOptions readOptions, PipelineOptions options) {
if (config.getValidate() && config.isDataAccessible() && readOptions.isDataAccessible()) {
- String tableId = checkNotNull(readOptions.getTableId().get());
+ ValueProvider<String> tableIdProvider = checkArgumentNotNull(readOptions.getTableId());
+ String tableId = checkArgumentNotNull(tableIdProvider.get());
try {
- checkArgument(
- getServiceFactory().checkTableExists(config, options, tableId),
- "Table %s does not exist",
- tableId);
+ boolean exists = getServiceFactory().checkTableExists(config, options, tableId);
+ checkArgument(exists, "Table %s does not exist", tableId);
} catch (IOException e) {
- LOG.warn("Error checking whether table {} exists; proceeding.", tableId, e);
+ throw new IllegalArgumentException(e);
Review Comment:
This is meant to catch exceptions when something went wrong and Bigtable client can't connect to the server. In this case we don't know for sure if the table exist or not (note that when the table doesn't exist the code will throw an error on line 695). So I'm not sure if throwing an exception is the right behavior here. But I don't think this will be a breaking change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #28288: Remove warning from catch in table exists validation
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28288:
URL: https://github.com/apache/beam/pull/28288#issuecomment-1703999661
Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
R: @robertwb for label java.
R: @chamikaramj for label io.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mutianf commented on a diff in pull request #28288: Remove warning from catch in table exists validation
Posted by "mutianf (via GitHub)" <gi...@apache.org>.
mutianf commented on code in PR #28288:
URL: https://github.com/apache/beam/pull/28288#discussion_r1330180787
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java:
##########
@@ -688,14 +688,13 @@ public final String toString() {
private void validateTableExists(
BigtableConfig config, BigtableReadOptions readOptions, PipelineOptions options) {
if (config.getValidate() && config.isDataAccessible() && readOptions.isDataAccessible()) {
- String tableId = checkNotNull(readOptions.getTableId().get());
+ ValueProvider<String> tableIdProvider = checkArgumentNotNull(readOptions.getTableId());
+ String tableId = checkArgumentNotNull(tableIdProvider.get());
try {
- checkArgument(
- getServiceFactory().checkTableExists(config, options, tableId),
- "Table %s does not exist",
- tableId);
+ boolean exists = getServiceFactory().checkTableExists(config, options, tableId);
+ checkArgument(exists, "Table %s does not exist", tableId);
} catch (IOException e) {
- LOG.warn("Error checking whether table {} exists; proceeding.", tableId, e);
+ throw new IllegalArgumentException(e);
Review Comment:
Can you make the exception more descriptive? Or log the error first? I also think we should re-throw IOException instead of changing it to a IllegalArgumentException because it's not an issue with the argument.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mutianf commented on a diff in pull request #28288: Remove warning from catch in table exists validation
Posted by "mutianf (via GitHub)" <gi...@apache.org>.
mutianf commented on code in PR #28288:
URL: https://github.com/apache/beam/pull/28288#discussion_r1330166796
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java:
##########
@@ -688,14 +688,13 @@ public final String toString() {
private void validateTableExists(
BigtableConfig config, BigtableReadOptions readOptions, PipelineOptions options) {
if (config.getValidate() && config.isDataAccessible() && readOptions.isDataAccessible()) {
- String tableId = checkNotNull(readOptions.getTableId().get());
+ ValueProvider<String> tableIdProvider = checkArgumentNotNull(readOptions.getTableId());
+ String tableId = checkArgumentNotNull(tableIdProvider.get());
try {
- checkArgument(
- getServiceFactory().checkTableExists(config, options, tableId),
- "Table %s does not exist",
- tableId);
+ boolean exists = getServiceFactory().checkTableExists(config, options, tableId);
+ checkArgument(exists, "Table %s does not exist", tableId);
} catch (IOException e) {
- LOG.warn("Error checking whether table {} exists; proceeding.", tableId, e);
+ throw new IllegalArgumentException(e);
Review Comment:
This is meant to catch exceptions when something went wrong and Bigtable client can't connect to the server. In this case we don't know for sure if the table exist or not (note that when the table doesn't exist the code will throw an error on line 695). So I'm not sure if throwing an exception is the right behavior here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #28288: Remove warning from catch in table exists validation
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28288:
URL: https://github.com/apache/beam/pull/28288#issuecomment-1703547234
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damondouglas commented on pull request #28288: Remove warning from catch in table exists validation
Posted by "damondouglas (via GitHub)" <gi...@apache.org>.
damondouglas commented on PR #28288:
URL: https://github.com/apache/beam/pull/28288#issuecomment-1703999516
assign set of reviewers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #28288: Remove warning from catch in table exists validation
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28288:
URL: https://github.com/apache/beam/pull/28288#issuecomment-1708516947
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damondouglas commented on a diff in pull request #28288: Remove warning from catch in table exists validation
Posted by "damondouglas (via GitHub)" <gi...@apache.org>.
damondouglas commented on code in PR #28288:
URL: https://github.com/apache/beam/pull/28288#discussion_r1330738321
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java:
##########
@@ -688,14 +688,13 @@ public final String toString() {
private void validateTableExists(
BigtableConfig config, BigtableReadOptions readOptions, PipelineOptions options) {
if (config.getValidate() && config.isDataAccessible() && readOptions.isDataAccessible()) {
- String tableId = checkNotNull(readOptions.getTableId().get());
+ ValueProvider<String> tableIdProvider = checkArgumentNotNull(readOptions.getTableId());
+ String tableId = checkArgumentNotNull(tableIdProvider.get());
try {
- checkArgument(
- getServiceFactory().checkTableExists(config, options, tableId),
- "Table %s does not exist",
- tableId);
+ boolean exists = getServiceFactory().checkTableExists(config, options, tableId);
+ checkArgument(exists, "Table %s does not exist", tableId);
} catch (IOException e) {
- LOG.warn("Error checking whether table {} exists; proceeding.", tableId, e);
+ throw new IllegalArgumentException(e);
Review Comment:
@mutianf Thank you again for your review and comments. Given the following, do you feel comfortable with the code changes?
1) We cannot throw an IOException as the overridden `public void validate(PipelineOptions options)` method cannot also throw this IOException. Therefore, we have to use a subclass of the RuntimeException, hence the use of IllegalArgumentException. The other choice would just be a RuntimeException. However, it seems to make sense that the arguments provided led to the Exception thrown.
2) The BigtableServiceFactory's checkTableExists already provides detail on the tableId and the originating ApiException. Therefore, the throw new IllegalArgumentException(e) would capture these details already.
In summary, I see the following scenarios:
| Scenario | Exception Message | Code Origin |
| --- | --- | --- |
| Not Found ApiException | Table <> does not exist | [BigtableServiceFactory.java:206](https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableServiceFactory.java#L206) |
| ApiException | "Error checking whether table <> exists" with ApiException details | [BigtableServiceFactory.java:210](https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableServiceFactory.java#L210) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org