You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by mohammed shambakey <sh...@gmail.com> on 2017/05/16 23:49:33 UTC
variable url for remote process group
Hi
Is it possible to use variables for remote process group, as the remote
server IP is not known in advance? If not, the ONLY alternative for remote
communication will be HTTP processors (e.g., handlehttprequest/response)
instead of site-to-site, right?
Regards
--
Mohammed
Re: variable url for remote process group
Posted by Bryan Bende <bb...@gmail.com>.
The RPG URL can be edited after importing to the next environment, and it should be ignored when comparing against the flow in registry so it won’t be seen as a change that needs to be committed. So although you can’t use a variable, you should still be able to version control a flow with an RPG and just update the URL after deployment with a script.
> On Oct 6, 2018, at 2:19 PM, mohammed shambakey <sh...@gmail.com> wrote:
>
> Hi
>
> Thanks, guys for your help. I ended up with building the whole RPG programmatically every time I need to change the URL, then delete (programmatically) the RPG at the end of the workflow
>
> Regards
>
> On Sat, Oct 6, 2018 at 4:37 AM Kevin Doran <kdoran.apache@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> Can you put the remote process group outside what is versioned to NiFi Registry? For example, if the remote process group is a sink, put everything upstream of it in a process group and version that as the thing that moves between environments, leaving the RPG, and the connection from the versioned PG to the RPG, as static in each environment. (Likewise, if the RPG is the source, put everything downstream of it in a process group that is versioned.)
>
> Hope this helps,
> Kevin
>
> On 10/5/18, 20:25, "evanthx" <evan@evan.org <ma...@evan.org>> wrote:
>
> i saw this and thought I'd reply as I have a similar issue.
>
> We have a staging and production cluster. I have a remote process group in
> them, and have our flow backed up in the registry as that seems to be the
> approved method for moving flows from staging to production.
>
> Since the remote process group can't be a variable, however, that means that
> we can't alter that variable between production and staging? So our
> production cluster keeps trying to send data to our staging cluster. We can
> fix it, but then the Registry is out of sync.
>
> We don't want to change it dynamically, for clarity - we just need a way to
> version our flow using the registry. If you have any advice I would
> appreciate it!
>
>
>
> --
> Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/ <http://apache-nifi-users-list.2361937.n4.nabble.com/>
>
>
>
> --
> Mohammed
Re: variable url for remote process group
Posted by mohammed shambakey <sh...@gmail.com>.
Hi
Thanks, guys for your help. I ended up with building the whole RPG
programmatically every time I need to change the URL, then delete
(programmatically) the RPG at the end of the workflow
Regards
On Sat, Oct 6, 2018 at 4:37 AM Kevin Doran <kd...@gmail.com> wrote:
> Hi,
>
> Can you put the remote process group outside what is versioned to NiFi
> Registry? For example, if the remote process group is a sink, put
> everything upstream of it in a process group and version that as the thing
> that moves between environments, leaving the RPG, and the connection from
> the versioned PG to the RPG, as static in each environment. (Likewise, if
> the RPG is the source, put everything downstream of it in a process group
> that is versioned.)
>
> Hope this helps,
> Kevin
>
> On 10/5/18, 20:25, "evanthx" <ev...@evan.org> wrote:
>
> i saw this and thought I'd reply as I have a similar issue.
>
> We have a staging and production cluster. I have a remote process
> group in
> them, and have our flow backed up in the registry as that seems to be
> the
> approved method for moving flows from staging to production.
>
> Since the remote process group can't be a variable, however, that
> means that
> we can't alter that variable between production and staging? So our
> production cluster keeps trying to send data to our staging cluster.
> We can
> fix it, but then the Registry is out of sync.
>
> We don't want to change it dynamically, for clarity - we just need a
> way to
> version our flow using the registry. If you have any advice I would
> appreciate it!
>
>
>
> --
> Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
>
>
--
Mohammed
Re: variable url for remote process group
Posted by Kevin Doran <kd...@gmail.com>.
Hi,
Can you put the remote process group outside what is versioned to NiFi Registry? For example, if the remote process group is a sink, put everything upstream of it in a process group and version that as the thing that moves between environments, leaving the RPG, and the connection from the versioned PG to the RPG, as static in each environment. (Likewise, if the RPG is the source, put everything downstream of it in a process group that is versioned.)
Hope this helps,
Kevin
On 10/5/18, 20:25, "evanthx" <ev...@evan.org> wrote:
i saw this and thought I'd reply as I have a similar issue.
We have a staging and production cluster. I have a remote process group in
them, and have our flow backed up in the registry as that seems to be the
approved method for moving flows from staging to production.
Since the remote process group can't be a variable, however, that means that
we can't alter that variable between production and staging? So our
production cluster keeps trying to send data to our staging cluster. We can
fix it, but then the Registry is out of sync.
We don't want to change it dynamically, for clarity - we just need a way to
version our flow using the registry. If you have any advice I would
appreciate it!
--
Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
Re: variable url for remote process group
Posted by evanthx <ev...@evan.org>.
i saw this and thought I'd reply as I have a similar issue.
We have a staging and production cluster. I have a remote process group in
them, and have our flow backed up in the registry as that seems to be the
approved method for moving flows from staging to production.
Since the remote process group can't be a variable, however, that means that
we can't alter that variable between production and staging? So our
production cluster keeps trying to send data to our staging cluster. We can
fix it, but then the Registry is out of sync.
We don't want to change it dynamically, for clarity - we just need a way to
version our flow using the registry. If you have any advice I would
appreciate it!
--
Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
Re: variable url for remote process group
Posted by Koji Kawamura <ij...@gmail.com>.
Hi Mohammed,
As RemoteProcessGroup and underlying Site-to-Site protocol maintains
connectivity between client and server, it is not supported to change
remote endpoint dynamically. HTTP processors may work but it's not
cluster aware so you will need a Load balancer in front of those.
If the number of possible end points is limited, you can setup RPG for
each of those and use RouteOnAttribute to route FlowFiles to desired
RPG.
Thanks,
Koji
On Wed, May 17, 2017 at 8:49 AM, mohammed shambakey
<sh...@gmail.com> wrote:
> Hi
>
> Is it possible to use variables for remote process group, as the remote
> server IP is not known in advance? If not, the ONLY alternative for remote
> communication will be HTTP processors (e.g., handlehttprequest/response)
> instead of site-to-site, right?
>
> Regards
>
> --
> Mohammed