[jira] [Commented] (BEAM-13162) BigQueryIO Storage Write API method - suppress logging API errors for known use cases.

["Beam JIRA Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514216#comment-17514216 ] 

Beam JIRA Bot commented on BEAM-13162:
--------------------------------------

This issue is P2 but has been unassigned without any comment for 60 days so it has been labeled "stale-P2". If this issue is still affecting you, we care! Please comment and remove the label. Otherwise, in 14 days the issue will be moved to P3.

Please see https://beam.apache.org/contribute/jira-priorities/ for a detailed explanation of what these priorities mean.


>  BigQueryIO Storage Write API method - suppress logging API errors for known use cases.
> ---------------------------------------------------------------------------------------
>
>                 Key: BEAM-13162
>                 URL: https://issues.apache.org/jira/browse/BEAM-13162
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-gcp
>    Affects Versions: 2.33.0
>            Reporter: Sergei Lilichenko
>            Priority: P2
>              Labels: stale-P2
>
> Many BigQuery Storage API errors get logged at ERROR level even though they are expected and handled by the Beam's transform. They should be suppressed, or output at DEBUG/INFO level to differentiate from abnormal behavior.
> Example:
> {code}
> {
>  "jsonPayload":
> { "message": "Got error io.grpc.StatusRuntimeException: ALREADY_EXISTS: The offset is within stream, expected offset 52125, received 51264 Entity: projects/event-processing-demo/datasets/bigquery_io/tables/events/streams/CiQ2MmZlOTFjNS0wMDAwLTIzNTItOWMxYS01ODI0MjlhOWRiOGM closing projects/event-processing-demo/datasets/bigquery_io/tables/events/streams/CiQ2MmZlOTFjNS0wMDAwLTIzNTItOWMxYS01ODI0MjlhOWRiOGM", "step": "Save Rows to BigQuery/StorageApiLoads/StorageApiWriteSharded/Write Records", "worker": "data-processing-streaming-10250846-kps9-harness-hnj3", "stage": "P6", "job": "2021-10-25_08_46_20-4863073108555028756", "thread": "258", "work": "3800026c09938fac-5cf2f68daacdd", "logger": "org.apache.beam.sdk.io.gcp.bigquery.StorageApiWritesShardedRecords" }
> ,
>  "resource": {
>  "type": "dataflow_step",
>  "labels":
> { "project_id": "event-processing-demo", "step_id": "Save Rows to BigQuery/StorageApiLoads/StorageApiWriteSharded/Write Records", "job_id": "2021-10-25_08_46_20-4863073108555028756", "region": "us-central1", "job_name": "data-processing-streaming-storage-write-api-200-2" }
> },
>  "timestamp": "2021-10-25T15:57:43.617Z",
>  "severity": "ERROR",
>  "labels":
> { "compute.googleapis.com/resource_type": "instance", "dataflow.googleapis.com/job_name": "data-processing-streaming-storage-write-api-200-2", "dataflow.googleapis.com/log_type": "supportability", "compute.googleapis.com/resource_id": "6365565973148954518", "dataflow.googleapis.com/job_id": "2021-10-25_08_46_20-4863073108555028756", "dataflow.googleapis.com/region": "us-central1", "compute.googleapis.com/resource_name": "data-processing-streaming-10250846-kps9-harness-hnj3" }
> ,
>  "logName": "projects/event-processing-demo/logs/dataflow.googleapis.com%2Fworker",
>  "receiveTimestamp": "2021-10-25T15:57:47.557344849Z"
>  }
> {code}
> Additional exceptions that fit this category:
>  * Got error {{io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Stream is closed due to com.google.api.gax.rpc.UnavailableException: io.grpc.StatusRuntimeException: UNAVAILABLE: Connection closed after GOAWAY. HTTP/2 error code: NO_ERROR, debug data: server_shutting_down closing}}
>  * Got error {{com.google.api.gax.rpc.AbortedException: io.grpc.StatusRuntimeException: ABORTED: Closing the stream because server is restarted}}. This is expected and client is advised to reconnect.
>  * Got error {{io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Stream is closed due to com.google.api.gax.rpc.AbortedException: io.grpc.StatusRuntimeException: ABORTED: Closing the stream because server is restarted}}. This is expected and client is advised to reconnect.
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)