You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Pierig Le Saux (JIRA)" <ji...@apache.org> on 2019/04/05 22:05:00 UTC

[jira] [Created] (BEAM-7022) BigQueryIO.write.withExtendedErrorInfo() does not catch missing tables (and maybe datasets)

Pierig Le Saux created BEAM-7022:
------------------------------------

             Summary: BigQueryIO.write.withExtendedErrorInfo()  does not catch missing tables (and maybe datasets)
                 Key: BEAM-7022
                 URL: https://issues.apache.org/jira/browse/BEAM-7022
             Project: Beam
          Issue Type: Bug
          Components: io-java-gcp
    Affects Versions: 2.11.0
            Reporter: Pierig Le Saux


In beam versions from 2.7.0 to 2.9.0 a missing table produces an exception in Dataflow that .getFailedInsertsWithErr() doesn't catch.

We get a GoogleJsonResponseException 404.


In beam versions 2.10.0 or 2.11.0 the step just gets silently stuck (logging every 5mins).

I would expect this error to be "catchable" just like a missing field in a table is.



I haven't checked the behaviour with a missing dataset yet, but I would like to be able to take action on this kind of error as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)