[jira] [Assigned] (BEAM-11226) Support larger tables in BigQuery storage API bounded source

["Kenneth Jung (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-11226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kenneth Jung reassigned BEAM-11226:
-----------------------------------

    Assignee: Kenneth Jung

> Support larger tables in BigQuery storage API bounded source
> ------------------------------------------------------------
>
>                 Key: BEAM-11226
>                 URL: https://issues.apache.org/jira/browse/BEAM-11226
>             Project: Beam
>          Issue Type: Improvement
>          Components: io-java-gcp
>    Affects Versions: 2.25.0
>            Reporter: Kenneth Jung
>            Assignee: Kenneth Jung
>            Priority: P2
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> The BigQuery storage API client specifies a default deadline of 10 minutes for calls to [CreateReadSession](https://cloud.google.com/bigquery/docs/reference/storage#create_a_session). However, for large tables, this operation can take longer than ten minutes, and as a result, read session creation simply fails for these tables. Some Beam runners, such as Google Cloud Dataflow, allow much longer for initial split calls on bounded sources -- 2 hours, in the case of Dataflow. We can support larger BigQuery tables with the storage API source by raising the client-side deadline for CreateReadSession to match this 2-hour limit.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)