[jira] [Created] (NIFI-11764) flowfile storage size mismatch

["Giovanni (Jira)" <ji...@apache.org>]

Giovanni created NIFI-11764:
-------------------------------

             Summary: flowfile storage size mismatch
                 Key: NIFI-11764
                 URL: https://issues.apache.org/jira/browse/NIFI-11764
             Project: Apache NiFi
          Issue Type: Improvement
            Reporter: Giovanni
         Attachments: image-2023-06-29-10-24-00-879.png, image-2023-06-29-10-25-06-665.png, image-2023-06-29-10-27-17-702.png, image-2023-06-29-10-28-45-547.png, image-2023-06-29-10-30-07-266.png, image-2023-06-29-10-31-33-526.png

Hi,

Nifi is reporting a wrong value for the flowfile storage used space.

If I check the dashboard it reports 528.05 MB on all nodes:

!image-2023-06-29-10-24-00-879.png|width=574,height=206!

The values are also confirmed by the nodes status history:

!image-2023-06-29-10-25-06-665.png|width=584,height=431!

However my monitoring tool reports 52KB only:

!image-2023-06-29-10-27-17-702.png|width=997,height=165!

These low values are confirmed on the hosts themselves:

!image-2023-06-29-10-28-45-547.png|width=485,height=179!

!image-2023-06-29-10-30-07-266.png|width=484,height=178!

!image-2023-06-29-10-31-33-526.png|width=485,height=176!

The flowfile repository settings are the same on all nodes:
 * nifi1

{code:java}
# FlowFile Repository
nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
nifi.flowfile.repository.wal.implementation=org.apache.nifi.wali.SequentialAccessWriteAheadLog
nifi.flowfile.repository.directory=/var/nifi/flowfile_repo/data
nifi.flowfile.repository.checkpoint.interval=20 secs
nifi.flowfile.repository.always.sync=false
nifi.flowfile.repository.retain.orphaned.flowfiles=truenifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
nifi.queue.swap.threshold=20000{code}

 * nifi2

{code:java}
# FlowFile Repository
nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
nifi.flowfile.repository.wal.implementation=org.apache.nifi.wali.SequentialAccessWriteAheadLog
nifi.flowfile.repository.directory=/var/nifi/flowfile_repo/data
nifi.flowfile.repository.checkpoint.interval=20 secs
nifi.flowfile.repository.always.sync=false
nifi.flowfile.repository.retain.orphaned.flowfiles=truenifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
nifi.queue.swap.threshold=20000 {code}

 * nifi3

{code:java}
# FlowFile Repository
nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
nifi.flowfile.repository.wal.implementation=org.apache.nifi.wali.SequentialAccessWriteAheadLog
nifi.flowfile.repository.directory=/var/nifi/flowfile_repo/data
nifi.flowfile.repository.checkpoint.interval=20 secs
nifi.flowfile.repository.always.sync=false
nifi.flowfile.repository.retain.orphaned.flowfiles=truenifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
nifi.queue.swap.threshold=20000 {code}

I also monitored the disk usage in realtime and it reports less than 100MB on each node for a very few seconds. However still far from the steady values reported in the GUI.

 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)