You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by dan young <da...@gmail.com> on 2020/02/20 23:18:49 UTC
zookeeper connection string question/clarification
Hello,
Using Nifi 1.11.1 in cluster mode with external zookeeper. Does the
nifi.zookeeper.connect string in the nifi.properties need to change from
say:
nifi.zookeeper.connect.string=
10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
nifi.zookeeper.connect.string=
10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
Changing the : to ; between the host and client port?
Re: zookeeper connection string question/clarification
Posted by dan young <da...@gmail.com>.
ok, I tried host;2181 and that doesn't seem to work. So I think host:2181
is still valid
On Thu, Feb 20, 2020 at 4:22 PM Andy LoPresto <al...@apache.org> wrote:
> Sorry, I should have elaborated that I was referencing the link from the
> MG. I realize you’re using external ZK and this is for embedded. Yes, I
> believe you will need to change the format of your connection string.
>
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <al...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Feb 20, 2020, at 3:20 PM, Andy LoPresto <al...@apache.org> wrote:
>
> Hi Dan,
>
> I believe the changes you’re looking for are here [1], copied below:
>
>
> - The Zookeeper dependency that NiFi uses for state management and
> cluster elections was upgraded to v3.5.5. From v3.5.x onwards, *Zookeeper
> changed the zookeeper.properties file format and as a result NiFi users
> using an existing embedded zookeeper will need to adjust their existing
> zookeeper.properties file accordingly*. More details here:
> https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport
> .
> For new deployments of the 1.10.0 release onwards, NiFi will be
> packaged with an updated template zookeeper.properties file.
> To update an existing zookeeper.properties file however, edit the
> conf/zookeeper.properties file:
> 1. Remove the clientPort=2181 line (or whatever your port number
> may be)
> 2. Add the client port to the end of the server string eg:
> server.1=localhost:2888:3888;2181
>
>
> [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <al...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Feb 20, 2020, at 3:18 PM, dan young <da...@gmail.com> wrote:
>
> Hello,
>
> Using Nifi 1.11.1 in cluster mode with external zookeeper. Does the
> nifi.zookeeper.connect string in the nifi.properties need to change from
> say:
>
> nifi.zookeeper.connect.string=
> 10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
>
> nifi.zookeeper.connect.string=
> 10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
>
> Changing the : to ; between the host and client port?
>
>
>
>
Re: zookeeper connection string question/clarification
Posted by Bryan Bende <bb...@gmail.com>.
I think this change only affects the server entries in
zookeeper.properties, so it’s mainly for people with existing embedded
zookeeper setups.
The nifi properties and state management entries didn’t change.
On Thu, Feb 20, 2020 at 6:45 PM Andy LoPresto <al...@apache.org> wrote:
> Thanks Dan. If it works, we can update the MG with that example as well.
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <al...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Feb 20, 2020, at 3:28 PM, dan young <da...@gmail.com> wrote:
>
> ok, great thank you. Yes, we're using external zookeeper; 3.5.6. I'm
> going to test this change out on a dev cluster real quick.
>
> On Thu, Feb 20, 2020 at 4:22 PM Andy LoPresto <al...@apache.org>
> wrote:
>
>> Sorry, I should have elaborated that I was referencing the link from the
>> MG. I realize you’re using external ZK and this is for embedded. Yes, I
>> believe you will need to change the format of your connection string.
>>
>>
>> Andy LoPresto
>> alopresto@apache.org
>> *alopresto.apache@gmail.com <al...@gmail.com>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>>
>> On Feb 20, 2020, at 3:20 PM, Andy LoPresto <al...@apache.org> wrote:
>>
>> Hi Dan,
>>
>> I believe the changes you’re looking for are here [1], copied below:
>>
>>
>> - The Zookeeper dependency that NiFi uses for state management and
>> cluster elections was upgraded to v3.5.5. From v3.5.x onwards, *Zookeeper
>> changed the zookeeper.properties file format and as a result NiFi users
>> using an existing embedded zookeeper will need to adjust their existing
>> zookeeper.properties file accordingly*. More details here:
>> https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport
>> .
>> For new deployments of the 1.10.0 release onwards, NiFi will be
>> packaged with an updated template zookeeper.properties file.
>> To update an existing zookeeper.properties file however, edit the
>> conf/zookeeper.properties file:
>> 1. Remove the clientPort=2181 line (or whatever your port number
>> may be)
>> 2. Add the client port to the end of the server string eg:
>> server.1=localhost:2888:3888;2181
>>
>>
>> [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
>>
>> Andy LoPresto
>> alopresto@apache.org
>> *alopresto.apache@gmail.com <al...@gmail.com>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>>
>> On Feb 20, 2020, at 3:18 PM, dan young <da...@gmail.com> wrote:
>>
>> Hello,
>>
>> Using Nifi 1.11.1 in cluster mode with external zookeeper. Does the
>> nifi.zookeeper.connect string in the nifi.properties need to change from
>> say:
>>
>> nifi.zookeeper.connect.string=
>> 10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
>>
>> nifi.zookeeper.connect.string=
>> 10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
>>
>> Changing the : to ; between the host and client port?
>>
>>
>>
>>
> --
Sent from Gmail Mobile
Re: zookeeper connection string question/clarification
Posted by Andy LoPresto <al...@apache.org>.
Thanks Dan. If it works, we can update the MG with that example as well.
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Feb 20, 2020, at 3:28 PM, dan young <da...@gmail.com> wrote:
>
> ok, great thank you. Yes, we're using external zookeeper; 3.5.6. I'm going to test this change out on a dev cluster real quick.
>
> On Thu, Feb 20, 2020 at 4:22 PM Andy LoPresto <alopresto@apache.org <ma...@apache.org>> wrote:
> Sorry, I should have elaborated that I was referencing the link from the MG. I realize you’re using external ZK and this is for embedded. Yes, I believe you will need to change the format of your connection string.
>
>
> Andy LoPresto
> alopresto@apache.org <ma...@apache.org>
> alopresto.apache@gmail.com <ma...@gmail.com>
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
>> On Feb 20, 2020, at 3:20 PM, Andy LoPresto <alopresto@apache.org <ma...@apache.org>> wrote:
>>
>> Hi Dan,
>>
>> I believe the changes you’re looking for are here [1], copied below:
>>
>> The Zookeeper dependency that NiFi uses for state management and cluster elections was upgraded to v3.5.5. From v3.5.x onwards, Zookeeper changed the zookeeper.properties file format and as a result NiFi users using an existing embedded zookeeper will need to adjust their existing zookeeper.properties file accordingly. More details here: https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport <https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport>.
>> For new deployments of the 1.10.0 release onwards, NiFi will be packaged with an updated template zookeeper.properties file.
>> To update an existing zookeeper.properties file however, edit the conf/zookeeper.properties file:
>> Remove the clientPort=2181 line (or whatever your port number may be)
>> Add the client port to the end of the server string eg: server.1=localhost:2888:3888;2181
>>
>> [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance <https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance>
>>
>> Andy LoPresto
>> alopresto@apache.org <ma...@apache.org>
>> alopresto.apache@gmail.com <ma...@gmail.com>
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>>
>>> On Feb 20, 2020, at 3:18 PM, dan young <danoyoung@gmail.com <ma...@gmail.com>> wrote:
>>>
>>> Hello,
>>>
>>> Using Nifi 1.11.1 in cluster mode with external zookeeper. Does the nifi.zookeeper.connect string in the nifi.properties need to change from say:
>>>
>>> nifi.zookeeper.connect.string=10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
>>>
>>> nifi.zookeeper.connect.string=10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
>>>
>>> Changing the : to ; between the host and client port?
>>
>
Re: zookeeper connection string question/clarification
Posted by dan young <da...@gmail.com>.
ok, great thank you. Yes, we're using external zookeeper; 3.5.6. I'm going
to test this change out on a dev cluster real quick.
On Thu, Feb 20, 2020 at 4:22 PM Andy LoPresto <al...@apache.org> wrote:
> Sorry, I should have elaborated that I was referencing the link from the
> MG. I realize you’re using external ZK and this is for embedded. Yes, I
> believe you will need to change the format of your connection string.
>
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <al...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Feb 20, 2020, at 3:20 PM, Andy LoPresto <al...@apache.org> wrote:
>
> Hi Dan,
>
> I believe the changes you’re looking for are here [1], copied below:
>
>
> - The Zookeeper dependency that NiFi uses for state management and
> cluster elections was upgraded to v3.5.5. From v3.5.x onwards, *Zookeeper
> changed the zookeeper.properties file format and as a result NiFi users
> using an existing embedded zookeeper will need to adjust their existing
> zookeeper.properties file accordingly*. More details here:
> https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport
> .
> For new deployments of the 1.10.0 release onwards, NiFi will be
> packaged with an updated template zookeeper.properties file.
> To update an existing zookeeper.properties file however, edit the
> conf/zookeeper.properties file:
> 1. Remove the clientPort=2181 line (or whatever your port number
> may be)
> 2. Add the client port to the end of the server string eg:
> server.1=localhost:2888:3888;2181
>
>
> [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <al...@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Feb 20, 2020, at 3:18 PM, dan young <da...@gmail.com> wrote:
>
> Hello,
>
> Using Nifi 1.11.1 in cluster mode with external zookeeper. Does the
> nifi.zookeeper.connect string in the nifi.properties need to change from
> say:
>
> nifi.zookeeper.connect.string=
> 10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
>
> nifi.zookeeper.connect.string=
> 10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
>
> Changing the : to ; between the host and client port?
>
>
>
>
Re: zookeeper connection string question/clarification
Posted by Andy LoPresto <al...@apache.org>.
Sorry, I should have elaborated that I was referencing the link from the MG. I realize you’re using external ZK and this is for embedded. Yes, I believe you will need to change the format of your connection string.
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Feb 20, 2020, at 3:20 PM, Andy LoPresto <al...@apache.org> wrote:
>
> Hi Dan,
>
> I believe the changes you’re looking for are here [1], copied below:
>
> The Zookeeper dependency that NiFi uses for state management and cluster elections was upgraded to v3.5.5. From v3.5.x onwards, Zookeeper changed the zookeeper.properties file format and as a result NiFi users using an existing embedded zookeeper will need to adjust their existing zookeeper.properties file accordingly. More details here: https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport <https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport>.
> For new deployments of the 1.10.0 release onwards, NiFi will be packaged with an updated template zookeeper.properties file.
> To update an existing zookeeper.properties file however, edit the conf/zookeeper.properties file:
> Remove the clientPort=2181 line (or whatever your port number may be)
> Add the client port to the end of the server string eg: server.1=localhost:2888:3888;2181
>
> [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance <https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance>
>
> Andy LoPresto
> alopresto@apache.org <ma...@apache.org>
> alopresto.apache@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
>> On Feb 20, 2020, at 3:18 PM, dan young <danoyoung@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hello,
>>
>> Using Nifi 1.11.1 in cluster mode with external zookeeper. Does the nifi.zookeeper.connect string in the nifi.properties need to change from say:
>>
>> nifi.zookeeper.connect.string=10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
>>
>> nifi.zookeeper.connect.string=10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
>>
>> Changing the : to ; between the host and client port?
>
Re: zookeeper connection string question/clarification
Posted by Andy LoPresto <al...@apache.org>.
Hi Dan,
I believe the changes you’re looking for are here [1], copied below:
The Zookeeper dependency that NiFi uses for state management and cluster elections was upgraded to v3.5.5. From v3.5.x onwards, Zookeeper changed the zookeeper.properties file format and as a result NiFi users using an existing embedded zookeeper will need to adjust their existing zookeeper.properties file accordingly. More details here: https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport <https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport>.
For new deployments of the 1.10.0 release onwards, NiFi will be packaged with an updated template zookeeper.properties file.
To update an existing zookeeper.properties file however, edit the conf/zookeeper.properties file:
Remove the clientPort=2181 line (or whatever your port number may be)
Add the client port to the end of the server string eg: server.1=localhost:2888:3888;2181
[1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance <https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance>
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Feb 20, 2020, at 3:18 PM, dan young <da...@gmail.com> wrote:
>
> Hello,
>
> Using Nifi 1.11.1 in cluster mode with external zookeeper. Does the nifi.zookeeper.connect string in the nifi.properties need to change from say:
>
> nifi.zookeeper.connect.string=10.xxx.x.xxx:2181,10.xxx.x.xxx:2181,10.xxx.x.xxx:2181
>
> nifi.zookeeper.connect.string=10.xxx.x.xxx;2181,10.xxx.x.xxx;2181,10.xxx.x.xxx;2181
>
> Changing the : to ; between the host and client port?