Which repositories would benefit from a higher performance disk?

[Eric Secules <es...@gmail.com>]

Hello,

I am working on configuring nifi to improve throughput. I already have each
of the repositories attached to their own Standard SSD disk. I am wondering
whether I am correct in thinking that I'd see the most improvement by
putting the content repository on a faster SSD SKU, or is it worth it to
put all the repositories on the fastest disks possible?

For reference my flows would like to handle a couple of millions of 25-50
KB flow files per hour without clustering. Right now it is able to do about
120,000 flowfiles per hour and the flow itself also has room for
improvement.

Thanks,
Eric

Re: Which repositories would benefit from a higher performance disk?

[Joe Witt <jo...@gmail.com>]

Eric

So assuming each ff is 50KB you currently see 1.6MB/sec for 120,000 ff/hr
and want to achieve at least 26MB/sec for 2,000,000 ff/hr.

These should be quite achievable rates for single nodes with modest
resources on a wide range of flow requirements.

Can you share a bit more about the cpu, memory, and flow setup?  How is
data received, what happens to it, where does it go?

Thanks

On Mon, Jan 16, 2023 at 8:53 AM Eric Secules <es...@gmail.com> wrote:

> Hello,
>
> I am working on configuring nifi to improve throughput. I already have
> each of the repositories attached to their own Standard SSD disk. I am
> wondering whether I am correct in thinking that I'd see the most
> improvement by putting the content repository on a faster SSD SKU, or is it
> worth it to put all the repositories on the fastest disks possible?
>
> For reference my flows would like to handle a couple of millions of 25-50
> KB flow files per hour without clustering. Right now it is able to do about
> 120,000 flowfiles per hour and the flow itself also has room for
> improvement.
>
> Thanks,
> Eric
>