You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by William Gosse <Wi...@aifoundry.com> on 2020/01/14 23:03:42 UTC
DistributedMapCacheService - automated creation
Is there any way to automate the creation of a DistributedMapCacheService
I was hoping I could do this via a template but it doesn’t to get exported the way the DistributedMapCacheClientService does.
The client got created just fine but there’s no service.
I don’t see away to do this with restful api either.
Any help would be greatly appreciated.
RE: DistributedMapCacheService - automated creation
Posted by William Gosse <Wi...@aifoundry.com>.
That was a very helpful suggestion. Here's what I came up with which seems to work:
#!/bin/bash
set -x
NIFI_ENCOMPASS_URL=http://xxx.xxx.xx.xxx:16080/nifi-api
ROOT_PROCESS=$(curl $NIFI_ENCOMPASS_URL/flow/process-groups/root)
ROOT_ID=`echo $ROOT_PROCESS | jq --raw-output '.processGroupFlow.id'`
echo "Nifi Encompass Connector Root ID = $ROOT_ID"
response=$(curl $NIFI_ENCOMPASS_URL/flow/process-groups/$ROOT_ID/controller-services)
controllerServices=`echo $response | jq --raw-output '.controllerServices'`
if [[ $controllerServices == '[]' ]]; then
echo "No existing controllerServices"
createService=true
else
if [[ $controllerServices == *"DistributedMapCacheServer"* ]]; then
echo "DistributedMapCacheServer exists"
else
echo "DistributedMapCacheServer does not exists"
createService=true
fi
fi
if [ "$createService" = true ]; then
clientId=$(curl $NIFI_ENCOMPASS_URL/flow/client-id)
curl -d '{"revision":{"clientId":"{$clientId}","version":0},"disconnectedNodeAcknowledged":false,"component":{"type":"org.apache.nifi.distributed.cache.server.map.DistributedMapCacheServer","bundle":{"group":"org.apache.nifi","artifact":"nifi-distributed-cache-services-nar","version":"1.10.0"}}}' -H "Content-Type: application/json" -X POST $NIFI_ENCOMPASS_URL/process-groups/$ROOT_ID/controller-services
fi
-----Original Message-----
From: Bryan Bende <bb...@gmail.com>
Sent: Wednesday, January 15, 2020 9:34 AM
To: users@nifi.apache.org
Subject: Re: DistributedMapCacheService - automated creation
[CAUTION: This email originated from outside of Kodak Alaris. Do not click links or open attachments unless you recognize the sender and know the content is safe.] ________________________________
Anything you can do from the UI goes through the REST API. You can open Chrome Dev Tools and see what calls are made to the server when you create the controller service from the UI.
On Wed, Jan 15, 2020 at 9:25 AM William Gosse <Wi...@aifoundry.com> wrote:
>
> Well we will be deploy Nifi in a docker container and need to configure everything at the time of the container’s creation and startup.
>
>
>
> To be specific I’m using DistributedMapCacheService with a DetectDuplicate process to track whether a flowfile with a same attribute value had been already created in the last 10 minutes. If so abandon the new flowfile.
>
>
>
> I guess the thin I liked about the DistributedMapCacheService is that it already exists in NiFi, no additional installation required.
>
>
>
>
>
> From: Mike Thomsen <mi...@gmail.com>
> Sent: Tuesday, January 14, 2020 7:42 PM
> To: users@nifi.apache.org
> Subject: Re: DistributedMapCacheService - automated creation
>
>
>
> [CAUTION: This email originated from outside of Kodak Alaris. Do not
> click links or open attachments unless you recognize the sender and
> know the content is safe.]
>
> ________________________________
>
> What is your use case? Because this sounds vaguely like something that you could implement implicitly with the HBase version by just creating one big cache table with an eye for not overlapping column qualifiers and possibly liberal use of HBase TTLs.
>
>
>
> On Tue, Jan 14, 2020 at 6:04 PM William Gosse <Wi...@aifoundry.com> wrote:
>
> Is there any way to automate the creation of a
> DistributedMapCacheService
>
>
>
> I was hoping I could do this via a template but it doesn’t to get exported the way the DistributedMapCacheClientService does.
>
>
>
> The client got created just fine but there’s no service.
>
>
>
> I don’t see away to do this with restful api either.
>
>
>
> Any help would be greatly appreciated.
>
>
>
>
Re: DistributedMapCacheService - automated creation
Posted by Bryan Bende <bb...@gmail.com>.
Anything you can do from the UI goes through the REST API. You can
open Chrome Dev Tools and see what calls are made to the server when
you create the controller service from the UI.
On Wed, Jan 15, 2020 at 9:25 AM William Gosse
<Wi...@aifoundry.com> wrote:
>
> Well we will be deploy Nifi in a docker container and need to configure everything at the time of the container’s creation and startup.
>
>
>
> To be specific I’m using DistributedMapCacheService with a DetectDuplicate process to track whether a flowfile with a same attribute value had been already created in the last 10 minutes. If so abandon the new flowfile.
>
>
>
> I guess the thin I liked about the DistributedMapCacheService is that it already exists in NiFi, no additional installation required.
>
>
>
>
>
> From: Mike Thomsen <mi...@gmail.com>
> Sent: Tuesday, January 14, 2020 7:42 PM
> To: users@nifi.apache.org
> Subject: Re: DistributedMapCacheService - automated creation
>
>
>
> [CAUTION: This email originated from outside of Kodak Alaris. Do not click links or open attachments unless you recognize the sender and know the content is safe.]
>
> ________________________________
>
> What is your use case? Because this sounds vaguely like something that you could implement implicitly with the HBase version by just creating one big cache table with an eye for not overlapping column qualifiers and possibly liberal use of HBase TTLs.
>
>
>
> On Tue, Jan 14, 2020 at 6:04 PM William Gosse <Wi...@aifoundry.com> wrote:
>
> Is there any way to automate the creation of a DistributedMapCacheService
>
>
>
> I was hoping I could do this via a template but it doesn’t to get exported the way the DistributedMapCacheClientService does.
>
>
>
> The client got created just fine but there’s no service.
>
>
>
> I don’t see away to do this with restful api either.
>
>
>
> Any help would be greatly appreciated.
>
>
>
>
RE: DistributedMapCacheService - automated creation
Posted by William Gosse <Wi...@aifoundry.com>.
Well we will be deploy Nifi in a docker container and need to configure everything at the time of the container’s creation and startup.
To be specific I’m using DistributedMapCacheService with a DetectDuplicate process to track whether a flowfile with a same attribute value had been already created in the last 10 minutes. If so abandon the new flowfile.
I guess the thin I liked about the DistributedMapCacheService is that it already exists in NiFi, no additional installation required.
From: Mike Thomsen <mi...@gmail.com>
Sent: Tuesday, January 14, 2020 7:42 PM
To: users@nifi.apache.org
Subject: Re: DistributedMapCacheService - automated creation
[CAUTION: This email originated from outside of Kodak Alaris. Do not click links or open attachments unless you recognize the sender and know the content is safe.]
________________________________
What is your use case? Because this sounds vaguely like something that you could implement implicitly with the HBase version by just creating one big cache table with an eye for not overlapping column qualifiers and possibly liberal use of HBase TTLs.
On Tue, Jan 14, 2020 at 6:04 PM William Gosse <Wi...@aifoundry.com>> wrote:
Is there any way to automate the creation of a DistributedMapCacheService
I was hoping I could do this via a template but it doesn’t to get exported the way the DistributedMapCacheClientService does.
The client got created just fine but there’s no service.
I don’t see away to do this with restful api either.
Any help would be greatly appreciated.
Re: DistributedMapCacheService - automated creation
Posted by Mike Thomsen <mi...@gmail.com>.
What is your use case? Because this sounds vaguely like something that you
could implement implicitly with the HBase version by just creating one big
cache table with an eye for not overlapping column qualifiers and possibly
liberal use of HBase TTLs.
On Tue, Jan 14, 2020 at 6:04 PM William Gosse <Wi...@aifoundry.com>
wrote:
> Is there any way to automate the creation of a DistributedMapCacheService
>
>
>
> I was hoping I could do this via a template but it doesn’t to get exported
> the way the DistributedMapCacheClientService does.
>
>
>
> The client got created just fine but there’s no service.
>
>
>
> I don’t see away to do this with restful api either.
>
>
>
> Any help would be greatly appreciated.
>
>
>
>
>
>