[jira] [Updated] (BEAM-11006) Allow Failsafe Handling of BigQuery Streaming Writes

["Kenneth Knowles (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-11006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kenneth Knowles updated BEAM-11006:
-----------------------------------
    Resolution: Fixed
        Status: Resolved  (was: Resolved)

Hello! Due to a bug in our Jira configuration, this issue had status:Resolved but resolution:Unresolved.

I am bulk editing these issues to have resolution:Fixed

If a different resolution is appropriate, please change it. To do this, click the "Resolve" button (you can do this even for closed issues) and set the Resolution field to the right value.

> Allow Failsafe Handling of BigQuery Streaming Writes
> ----------------------------------------------------
>
>                 Key: BEAM-11006
>                 URL: https://issues.apache.org/jira/browse/BEAM-11006
>             Project: Beam
>          Issue Type: Improvement
>          Components: extensions-java-gcp
>            Reporter: Dylan Hercher
>            Priority: P2
>              Labels: Clarified, Done, bigquery, google-cloud-bigquery
>   Original Estimate: 336h
>          Time Spent: 2.5h
>  Remaining Estimate: 333.5h
>
> To allow handling of a generic failsafe (of any type) would allow a dead letter queue to retain the original source data rather than the cleaned version and could be more easily understood and re-processed.
>  
> The BigQueryIO.Write currently supports `withFormatFunction` which allows for a serializable function to be applied to each datapoint -> TableRow.  Ideally that same source value could be converted with a separate function:
> `withFailsafeFormatFunction` taken (InputT -> TableRow) or possibly (InputT -> OutputT), though the backwards compatibility of OutputT is more difficult.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)