You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Ravneet Cheema <ra...@macrohealth.com> on 2021/08/18 03:01:34 UTC
Persistence in DistributedMapCacheServer
Hello ,
I am using DistributedMapCacheServer with setting of Persistent Directory. I can see contents of cache in the directory, but the contents do not survive nifi restart .
Older values from cache should be available after restart, right ?
I have attached template where I have configured client and distributed map cache server.
Based on answer in following link , data in cache should be available on restart.
https://community.cloudera.com/t5/Support-Questions/How-to-update-nifi-cache-on-nifi-service-restart/td-p/225434
Please provide insights on how I can achieve a persistent cache in which data survives pod restart .
Thanks,
Ravneet
Re: Persistence in DistributedMapCacheServer
Posted by Ravneet Cheema <ra...@macrohealth.com>.
Thank you so much for the reply Chris.
Yes, I am deploying NiFi into Kubernetes. I am using a directory backed by persistent volume but after every pod restart new .journal file is created and older values of cache are lost .
Just for testing, I pointed Persistent Directory to a path in content_repository as persistence is enabled and content_repository’s data survives pod restart as it is backed by persistent volume, but I can see all other content repository specific data surviving pod restart but cache .journal file is newly created after pod restart loosing older values in cache.
Please point out if I am missing anything in achieving a persistent cache in which data survives pod restart.
Thanks,
Ravneet
From: Chris Sampson <ch...@naimuri.com>
Date: Wednesday, August 18, 2021 at 12:55 AM
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Re: Persistence in DistributedMapCacheServer
CAUTION: This email originated from outside of the organization. Do not click any links or open any attachments unless you trust the sender and know the content is safe.
You mention "Pod restart", does this mean you're deploying NiFi into Kubernetes? If so, you will need to mount an external Persistent Volume into the Pod as a directory that you can use for the Persistent Directory of the DistributedMapCacheServer.
We've used NiFi Kubernetes and successfully persisted DMC data between Pod restarts using this approach.
---
Chris Sampson
IT Consultant
chris.sampson@naimuri.com<ma...@naimuri.com>
[https://docs.google.com/uc?export=download&id=1oPtzd0P7DqtuzpjiTRAa6h6coFitpqom&revid=0B9aXwC5rMc6lVlZ2OWpUaVlFVmUwTlZBdjQ0KzAxb1dZS2hJPQ]<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.naimuri.com%2F&data=04%7C01%7Cravneet.cheema%40macrohealth.com%7C68a34fc4975e498ab02508d9621d65cb%7C8c80ec49c9934513a1883c8e08d31cd6%7C0%7C0%7C637648701451657436%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=yGAhGRJRf4ky36CSFU6K6GC3p7zPePDq1lCr%2FcTT7IM%3D&reserved=0>
On Wed, 18 Aug 2021 at 04:01, Ravneet Cheema <ra...@macrohealth.com>> wrote:
Hello ,
I am using DistributedMapCacheServer with setting of Persistent Directory. I can see contents of cache in the directory, but the contents do not survive nifi restart .
Older values from cache should be available after restart, right ?
I have attached template where I have configured client and distributed map cache server.
Based on answer in following link , data in cache should be available on restart.
https://community.cloudera.com/t5/Support-Questions/How-to-update-nifi-cache-on-nifi-service-restart/td-p/225434<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcommunity.cloudera.com%2Ft5%2FSupport-Questions%2FHow-to-update-nifi-cache-on-nifi-service-restart%2Ftd-p%2F225434&data=04%7C01%7Cravneet.cheema%40macrohealth.com%7C68a34fc4975e498ab02508d9621d65cb%7C8c80ec49c9934513a1883c8e08d31cd6%7C0%7C0%7C637648701451667395%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=emAXUMJ8Oxu%2FPbn1H95MYQxdnPXL%2BGQvDSbWxXm2ZCU%3D&reserved=0>
Please provide insights on how I can achieve a persistent cache in which data survives pod restart .
Thanks,
Ravneet
Re: Persistence in DistributedMapCacheServer
Posted by Chris Sampson <ch...@naimuri.com>.
You mention "Pod restart", does this mean you're deploying NiFi into
Kubernetes? If so, you will need to mount an external Persistent Volume
into the Pod as a directory that you can use for the Persistent Directory
of the DistributedMapCacheServer.
We've used NiFi Kubernetes and successfully persisted DMC data between Pod
restarts using this approach.
---
*Chris Sampson*
IT Consultant
chris.sampson@naimuri.com
<https://www.naimuri.com/>
On Wed, 18 Aug 2021 at 04:01, Ravneet Cheema <ra...@macrohealth.com>
wrote:
> Hello ,
>
>
>
> I am using DistributedMapCacheServer with setting of Persistent Directory.
> I can see contents of cache in the directory, but the contents do not
> survive nifi restart .
>
> Older values from cache should be available after restart, right ?
>
> I have attached template where I have configured client and distributed
> map cache server.
>
> Based on answer in following link , data in cache should be available on
> restart.
>
> https://community.cloudera.com/t5/Support-Questions/How-to-update-nifi-cache-on-nifi-service-restart/td-p/225434
>
> Please provide insights on how I can achieve a persistent cache in which
> data survives pod restart .
>
>
>
> Thanks,
>
> Ravneet
>
>
>