You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Martijn Dekkers (Jira)" <ji...@apache.org> on 2021/03/09 06:44:00 UTC
[jira] [Commented] (NIFI-7631) Create
nifi.content.repository.implementation for SharedFileSystem and
nifi.flowfile.repository.implementation for Apache Pulsar
[ https://issues.apache.org/jira/browse/NIFI-7631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17297863#comment-17297863 ]
Martijn Dekkers commented on NIFI-7631:
---------------------------------------
Hi [~ryanrlamothe] we are working on something similar. Do you have Github for this, and/or are you open to collaborate?
> Create nifi.content.repository.implementation for SharedFileSystem and nifi.flowfile.repository.implementation for Apache Pulsar
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-7631
> URL: https://issues.apache.org/jira/browse/NIFI-7631
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Core Framework
> Reporter: Ryan LaMothe
> Priority: Major
>
> I would like to begin the development of a new nifi.content.repository.implementation that utilizes a shared filesystem and a new nifi.flowfile.repository.implementation that utilizes Apache Pulsar. In our modern, cloud-based streaming message environments, we are using high performance shared filesystems for persistent file/object storage and Apache Pulsar for message management. Apache NiFi currently supports only local disk (non-volatile) and in-memory (volatile) repository implementations. This means that Apache NiFi currently performs double duty as both a workflow management environment and a message/data management system, as there are no shared data management or remote message management repository implementations available.
> The proposed new feature development would create a new content repository implementation designed around a shared filesystem and a new flowfile implementation designed around a distributed message bus. In essence, replacing the concept of "NiFi local queues" with "shared filesystem storage and remote flowfile queues". This work will support NiFi as a pure workflow management environment, decoupling it from its current data and message management responsibilities.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)