You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Carlos Alonso <ca...@mrcalonso.com> on 2018/03/27 15:38:27 UTC

BigQuery FILE_LOADS recover from errors

Hi all!!

On my pipeline I want to dump some data into BQ using FILE_LOADS write
method and I can't see how would I recover from errors (i.e. on the
pipeline detect which records couldn't be inserted and store it somewhere
else for further inspection) as the WriteTables transform throws an
Exception after BatchLoads.MAX_RETRY_JOBS retries...

How could I approach that?

Thanks!