[jira] [Comment Edited] (BEAM-13376) Missing error for nonexistent column family BigTable

["Ning (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-13376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17491183#comment-17491183 ] 

Ning edited comment on BEAM-13376 at 2/11/22, 10:30 PM:
--------------------------------------------------------

[~wangrobert] In that case, for this ticket, it doesn't sound like a bug in the bigtable client. The client is just not used as intended.

In BEAM-13606, the bug of the bigtable client is very generic and could happen to any code that invokes row mutation, including their own status for-loop code snippet.

Things are wrong on bigtable client side involves:
* the client suppresses retryable errors, causing data loss because the invoker (Beam I/O) doesn't know which rows are failed.
* when retryable errors are suppressed, None is returned instead of Status. So the for-loop in the [sample code](https://github.com/googleapis/python-bigtable/blob/main/samples/snippets/writes/write_batch.py#L40) will explode because it tries to access `None.code`.
* the client's worker raises those later suppressed retryable errors. But note a retryable error (such as deadline_exceeded or unavailable) for a row in a batch pollutes other rows in that batch: other rows could be successful or running into non-retryable errors. But now a retryable error is raised, the worker try-except-breaks at that point and retries the whole batch; eventually the worker gives up, bubbles up the error, suppresses it and return None for all rows in that batch: ending in a corrupted state for the whole system.



was (Author: ningk):
[~wangrobert] In that case, for this ticket, it doesn't sound like a bug in the bigtable client. The client is just not used as intended.

In BEAM-13606, the bug of the bigtable client is very generic and could happen to any code that invokes row mutation, including their own status for-loop code snippet.

Things are wrong involves:
* the client suppresses retryable errors, causing data loss because the invoker (Beam I/O) doesn't know which rows are failed.
* when retryable errors are suppressed, None is returned instead of Status. So the for-loop in the [sample code](https://github.com/googleapis/python-bigtable/blob/main/samples/snippets/writes/write_batch.py#L40) will explode because it tries to access `None.code`.
* the client's worker raises those later suppressed retryable errors. But note a retryable error (such as deadline_exceeded or unavailable) for a row in a batch pollutes other rows in that batch: other rows could be successful or running into non-retryable errors. But now a retryable error is raised, the worker try-except-breaks at that point and retries the whole batch; eventually the worker gives up, bubbles up the error, suppresses it and return None for all rows in that batch: ending in a corrupted state for the whole system.


> Missing error for nonexistent column family BigTable
> ----------------------------------------------------
>
>                 Key: BEAM-13376
>                 URL: https://issues.apache.org/jira/browse/BEAM-13376
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-gcp
>            Reporter: PierreOberholzer
>            Priority: P1
>
> Currently, there is no error thrown by BigTable in case the Column Families are not defined at writing time. That is a misleading behavior as the user believes the job has completed, though with empty table.
> A bug was raised on BigTable:
> [https://issuetracker.google.com/issues/186053077?pli=1]
> But it should be made sure that the Beam IO will log this error appropriately.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)