You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@nifi.apache.org by "Pierre Villard (Jira)" <ji...@apache.org> on 2025/04/10 13:11:00 UTC

[jira] [Created] (NIFI-14461) PutBigQuery - re-use of the writer stream

Pierre Villard created NIFI-14461:
-------------------------------------

             Summary: PutBigQuery - re-use of the writer stream
                 Key: NIFI-14461
                 URL: https://issues.apache.org/jira/browse/NIFI-14461
             Project: Apache NiFi
          Issue Type: Improvement
          Components: Extensions
            Reporter: Pierre Villard


At the moment, PutBigQuery creates one WriterStream per flow file. This not efficient and it would be better to have proper management of a map of destination table / writer stream (because of expression language support) with proper handling of the offset in order to reuse the same streams across many flow files. That would also be an opportunity to support concurrent tasks for better performances/

https://cloud.google.com/bigquery/docs/write-api-best-practices#connection_pool_management



--
This message was sent by Atlassian Jira
(v8.20.10#820010)