[jira] [Commented] (NIFI-12442) Adding support for RocksDB

["David Handermann (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-12442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17791703#comment-17791703 ] 

David Handermann commented on NIFI-12442:
-----------------------------------------

Thanks for raising the question on this potential new feature [~kommpn].

Is the idea that these components would support RocksDB in the local file system? If that is the primary use case, this seems less useful as a general component, since it would require each NiFi node to maintain access to reference RocksDB file.

If you could outline some potential use cases, that would be helpful.

This might be a case where it would be worth developing the components in a separate repository, which could allow users to evaluate the components before bringing them under maintenance of the Apache NiFi project itself.

> Adding support for RocksDB
> --------------------------
>
>                 Key: NIFI-12442
>                 URL: https://issues.apache.org/jira/browse/NIFI-12442
>             Project: Apache NiFi
>          Issue Type: Improvement
>    Affects Versions: 1.23.2
>            Reporter: Giovanni
>            Priority: Minor
>   Original Estimate: 336h
>  Remaining Estimate: 336h
>
> I would like to suggest the creation of 3 new components:
> The first one is a service, which opens an existing RocksDb or, eventually, using RocksOptions, create it from scratch. It will manage all the open options (classic, read/write, only read, secondary).
> The second one, is a RocksDbReader, that uses the service to communicate with the RocksDb in order to retrieve informations through a lookup. It can save the searched content inside an attribute or inside the flowFile content. It will be capable of using both APIs, such as simple "db.get" and via RocksIterator.
> The last one, is a RocksDbWriter, that uses the same service as the reader, but can write values inside the RocksDb, both from flowFile attribute or flowFile content, using properties to determine the key to use.
>  
> Feel free to express your opinions, if you think this will be useful or useless.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)