[jira] [Commented] (BEAM-13849) Introduce proper retry handling for Python BigTable I/O

["Yi Hu (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-13849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17508861#comment-17508861 ] 

Yi Hu commented on BEAM-13849:
------------------------------

Currently Python BigTable I/O calls python-bigtable [mutation_rows|https://github.com/googleapis/python-bigtable/blob/ff7f1901b6420e66e1388e757eeec20d30484ad9/google/cloud/bigtable/table.py#L719] to proceed batch row mutation. The latter function call already contains retry handling for retry-able errors until time limit has exceeded. The returned status_list, if containing non-zero status code (or None), it means the error is either not retry-able or retry has exceeded the default time limit.

Looks like the retry mechanism existed in python-bigtable api suffices, and the current code handles the error properly.

In the future if we migrate to portable framework (instead of calling python specific bigtable api here), we may need to similarly handle the retry-able errors in a unified way across multi-language sdks.

> Introduce proper retry handling for Python BigTable I/O
> -------------------------------------------------------
>
>                 Key: BEAM-13849
>                 URL: https://issues.apache.org/jira/browse/BEAM-13849
>             Project: Beam
>          Issue Type: Improvement
>          Components: io-py-gcp
>            Reporter: Chamikara Madhusanka Jayalath
>            Assignee: Yi Hu
>            Priority: P2
>              Labels: newbie, starter
>
> Currently we only have a sink and the implementation is here: https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/bigtableio.py
> For example, we should properly identify various errors codes and retry failed mutations appropriately.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)