[jira] [Created] (NIFI-11449) add autocommit property to control commit in PutDatabaseRecord processor

["Abdelrahim Ahmad (Jira)" <ji...@apache.org>]

Abdelrahim Ahmad created NIFI-11449:
---------------------------------------

             Summary: add autocommit property to control commit in PutDatabaseRecord processor
                 Key: NIFI-11449
                 URL: https://issues.apache.org/jira/browse/NIFI-11449
             Project: Apache NiFi
          Issue Type: Improvement
          Components: Extensions
    Affects Versions: 1.21.0
         Environment: Any Nifi Deployment
            Reporter: Abdelrahim Ahmad


The issue is with the {{PutDatabaseRecord}} processor in Apache NiFi. When using the processor with the Trino-JDBC-Driver to write to an Iceberg catalog, it disables the autocommit feature. This leads to errors such as "{*}Catalog only supports writes using autocommit: iceberg{*}".

To fix this issue, the autocommit feature needs to be added in the processor to be enabled/disabled.
enabling auto-commit in the Nifi PutDatabaseRecord processor is important for Deltalake, Iceberg, and Hudi as it ensures data consistency and integrity by allowing atomic writes to be performed in the underlying database. This will allow the process to be widely used with bigger range of databases.

P.S.: using PutSQL is not a good option at all due to the sensitivity of these tables when dealing with small inserts.

Thanks and best regards
Abdelrahim Ahmad



--
This message was sent by Atlassian Jira
(v8.20.10#820010)