You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Takashi Menjo (Jira)" <ji...@apache.org> on 2021/02/12 02:57:00 UTC
[jira] [Updated] (NIFI-8219) PMEM-backed Repositories
[ https://issues.apache.org/jira/browse/NIFI-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Takashi Menjo updated NIFI-8219:
--------------------------------
Description:
*Please note that this patchset works only on x64 Linux (4.15 or later) for now.*
Persistent memory (PMEM) is non-volatile and byte-addressable memory installed into DIMM slots. With Filesystem DAX (Direct Access) and PMDK (Persistent Memory Development Kit), a program can map on-PMEM files to userspace then read and write its data, bypassing page caches. These technologies could bring you better I/O performance than traditional disks.
I would propose a patchset that lets FlowFile, Content, and Provenance Repositories use PMDK (via JNI) to write their data to their files shown as follows:
* FlowFile Repository: Journals (.journal)
* Content Repository: Content/Resource Claims
* Provenance Repository: Provenance logs (.prov) and ToCs (.toc)
Prerequisites on compile: gcc, libpmem-devel (1.5 or later; 1.9 or later recommended)
Prerequisites on run: libpmem (1.5 or later; 1.9 or later recommended)
was:
Persistent memory (PMEM) is non-volatile and byte-addressable memory installed into DIMM slots. With Filesystem DAX (Direct Access) and PMDK (Persistent Memory Development Kit), a program can map on-PMEM files to userspace then read and write its data, bypassing page caches. These technologies could bring you better I/O performance than traditional disks.
I would propose a patchset that lets FlowFile, Content, and Provenance Repositories use PMDK (via JNI) to write their data to their files shown as follows:
* FlowFile Repository: Journals (.journal)
* Content Repository: Content/Resource Claims
* Provenance Repository: Provenance logs (.prov) and ToCs (.toc)
*Please note that this patchset works only on x64 Linux (4.15 or later) for now.*
> PMEM-backed Repositories
> ------------------------
>
> Key: NIFI-8219
> URL: https://issues.apache.org/jira/browse/NIFI-8219
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Core Framework
> Environment: PMEM, x64, Linux, DAX, PMDK
> Reporter: Takashi Menjo
> Priority: Minor
>
> *Please note that this patchset works only on x64 Linux (4.15 or later) for now.*
> Persistent memory (PMEM) is non-volatile and byte-addressable memory installed into DIMM slots. With Filesystem DAX (Direct Access) and PMDK (Persistent Memory Development Kit), a program can map on-PMEM files to userspace then read and write its data, bypassing page caches. These technologies could bring you better I/O performance than traditional disks.
> I would propose a patchset that lets FlowFile, Content, and Provenance Repositories use PMDK (via JNI) to write their data to their files shown as follows:
> * FlowFile Repository: Journals (.journal)
> * Content Repository: Content/Resource Claims
> * Provenance Repository: Provenance logs (.prov) and ToCs (.toc)
> Prerequisites on compile: gcc, libpmem-devel (1.5 or later; 1.9 or later recommended)
> Prerequisites on run: libpmem (1.5 or later; 1.9 or later recommended)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)