[GitHub] [beam] ihji commented on a change in pull request #12822: [BEAM-10880] Log error counts to debug BigQuery streaming insert requ…

[GitBox <gi...@apache.org>]

ihji commented on a change in pull request #12822:
URL: https://github.com/apache/beam/pull/12822#discussion_r489081111



##########
File path: sdks/python/apache_beam/io/gcp/bigquery_tools.py
##########
@@ -538,6 +539,13 @@ def _insert_all_rows(
     try:
       response = self.client.tabledata.InsertAll(request)
       # response.insertErrors is not [] if errors encountered.
+    except HttpError as exn:
+      if error_counter:
+        content = json.loads(exn.content)
+        error_counter.record(
+            '%s(%s)' %
+            (content['error']['errors'][0]['reason'], exn.status_code))

Review comment:
       It records string error codes and numeric http codes from BigQuery API JSON response https://cloud.google.com/bigquery/docs/error-messages. For example:
   ```
   {quotaExceeded(403): 461, rateLimitExceeded(403): 2724}
   ```
   I think string error codes here are human readable and clearly shows the cause of errors. Do you want to append any other string messages to them?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org