You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by "Rosso, Roland" <Ro...@AdventHealth.com> on 2020/07/08 19:22:26 UTC
RE: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
Thank you Bryan. Missed this link.
Would you have a recommended path to upgrade from 0.1.0 keeping H2?
Thank you
Roland
From: Bryan Bende <bb...@gmail.com>
Sent: Wednesday, July 8, 2020 3:10 PM
To: users@nifi.apache.org
Subject: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
Hello,
Older releases can always be found in the apache archive:
https://archive.apache.org/dist/nifi/nifi-registry/
Thanks,
Bryan
On Wed, Jul 8, 2020 at 2:14 PM Rosso, Roland <Ro...@adventhealth.com>> wrote:
Hi all,
We are upgrading NiFi from 1.9.2 to 1.11.4. Our 1.9.2 uses NiFi Registry 0.1.0 still and we have some catching up to do.
After reading the migration guidance for NiFi registry, I thought the prudent approach would be to upgrade to 0.2.0, then up from there.
I cannot find the 0.2.0 install binaries; I found the source code but would rather not compile it.
Could you tell me if there is a safe path from Registry 0.1.0 to 0.5.0 (or even 0.6.0)? We would likely keep H2 initially and secure it.
We’re on Cloudera CDH 6.2.
Thank you
Roland Rosso
This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.
This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.
RE: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
Posted by "Rosso, Roland" <Ro...@AdventHealth.com>.
Yes sir,
Same buckets and flows are available. From the start on 0.6.0, I was able to see the buckets in Registry and via NiFi --> Import. However, I did have to copy over the flow_storage directory to be able to import a flow (otherwise it would show the process group id and say unavailable, something to that effect.
I followed the directions as you have below, point to old, checked it created the new nifi-registry-primary.mv.db. Since then, I’ve done a couple of Registry stop/start, and removed the ‘old’ db entries from nifi-registry.properties, since we’re on 0.6.0 now.
All seems ok, but the old db shows as 2.4M, the new is merely 152K.
From: Bryan Bende <bb...@gmail.com>
Sent: Friday, July 10, 2020 2:35 PM
To: users@nifi.apache.org
Subject: Re: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
Do you have all the same buckets and flows in the registry UI? If so then I would think it worked correctly.
You should have populated both sets of these properties to migrate the old H2 to the new H2...
# legacy database properties, used to migrate data from original DB to new DB below
# NOTE: Users upgrading from 0.1.0 should leave these populated, but new installs after 0.1.0 should leave these empty
nifi.registry.db.directory=
nifi.registry.db.url.append=
# database properties
nifi.registry.db.url=jdbc:h2:./database/nifi-registry-primary;AUTOCOMMIT=OFF;DB_CLOSE_ON_EXIT=FALSE;LOCK_MODE=3;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
nifi.registry.db.driver.class=org.h2.Driver
On Fri, Jul 10, 2020 at 1:53 PM Rosso, Roland <Ro...@adventhealth.com>> wrote:
Hi Bryan,
I was able to go from 0.1.0 to 0.6.0. I think the database upgraded fine but the ‘new’ H2 database nifi-registry-primary.mv.db is only about 1/10 the size of the old one.
I also had to copy the flow_storage directory from 0.1.0 to 0.6.0 (did a test upgrade in parallel), but it all appears to work fine after I did that. DB size still has me concerned though.
I may have fumbled something in the steps and if you can point it out, that’d be great.
Thank you.
Roland
From: Bryan Bende <bb...@gmail.com>>
Sent: Wednesday, July 8, 2020 4:04 PM
To: users@nifi.apache.org<ma...@nifi.apache.org>
Subject: Re: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
I'm pretty sure you can go directly from 0.1.0 to 0.6.0, there is no requirement to switch from H2 to Postgres, that was just an example in the migration for people that wanted it.
The change that was relevant was between 0.1.0 and 0.2.0, the database properties were refactored, and even if you are using H2, it will migrate from the old H2 to a new one, so just make sure you follow that guidance...
"The 0.2.0 release introduces new database configuration in nifi-registry.properties which provides more flexible options. When upgrading from 0.1.0, ensure that nifi.registry.db.directory points to the location of the existing H2 database to ensure it is migrated to the new database. Please consult the Database Properties<https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#database-properties> section of the Admin Guide for further details of these properties."
On Wed, Jul 8, 2020 at 3:22 PM Rosso, Roland <Ro...@adventhealth.com>> wrote:
Thank you Bryan. Missed this link.
Would you have a recommended path to upgrade from 0.1.0 keeping H2?
Thank you
Roland
From: Bryan Bende <bb...@gmail.com>>
Sent: Wednesday, July 8, 2020 3:10 PM
To: users@nifi.apache.org<ma...@nifi.apache.org>
Subject: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
Hello,
Older releases can always be found in the apache archive:
https://archive.apache.org/dist/nifi/nifi-registry/
Thanks,
Bryan
On Wed, Jul 8, 2020 at 2:14 PM Rosso, Roland <Ro...@adventhealth.com>> wrote:
Hi all,
We are upgrading NiFi from 1.9.2 to 1.11.4. Our 1.9.2 uses NiFi Registry 0.1.0 still and we have some catching up to do.
After reading the migration guidance for NiFi registry, I thought the prudent approach would be to upgrade to 0.2.0, then up from there.
I cannot find the 0.2.0 install binaries; I found the source code but would rather not compile it.
Could you tell me if there is a safe path from Registry 0.1.0 to 0.5.0 (or even 0.6.0)? We would likely keep H2 initially and secure it.
We’re on Cloudera CDH 6.2.
Thank you
Roland Rosso
This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.
This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.
This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.
This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.
Re: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
Posted by Bryan Bende <bb...@gmail.com>.
Do you have all the same buckets and flows in the registry UI? If so then I
would think it worked correctly.
You should have populated both sets of these properties to migrate the old
H2 to the new H2...
# legacy database properties, used to migrate data from original DB to new
DB below
# NOTE: Users upgrading from 0.1.0 should leave these populated, but new
installs after 0.1.0 should leave these empty
nifi.registry.db.directory=
nifi.registry.db.url.append=
# database properties
nifi.registry.db.url=jdbc:h2:./database/nifi-registry-primary;AUTOCOMMIT=OFF;DB_CLOSE_ON_EXIT=FALSE;LOCK_MODE=3;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
nifi.registry.db.driver.class=org.h2.Driver
On Fri, Jul 10, 2020 at 1:53 PM Rosso, Roland <Ro...@adventhealth.com>
wrote:
> Hi Bryan,
>
> I was able to go from 0.1.0 to 0.6.0. I think the database upgraded fine
> but the ‘new’ H2 database nifi-registry-primary.mv.db is only about 1/10
> the size of the old one.
>
> I also had to copy the flow_storage directory from 0.1.0 to 0.6.0 (did a
> test upgrade in parallel), but it all appears to work fine after I did
> that. DB size still has me concerned though.
>
> I may have fumbled something in the steps and if you can point it out,
> that’d be great.
>
> Thank you.
>
> Roland
>
> *From:* Bryan Bende <bb...@gmail.com>
> *Sent:* Wednesday, July 8, 2020 4:04 PM
> *To:* users@nifi.apache.org
> *Subject:* Re: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
>
>
>
> I'm pretty sure you can go directly from 0.1.0 to 0.6.0, there is no
> requirement to switch from H2 to Postgres, that was just an example in the
> migration for people that wanted it.
>
>
>
> The change that was relevant was between 0.1.0 and 0.2.0, the database
> properties were refactored, and even if you are using H2, it will migrate
> from the old H2 to a new one, so just make sure you follow that guidance...
>
>
>
> "The 0.2.0 release introduces new database configuration in *
> nifi-registry.properties* which provides more flexible options. When
> upgrading from 0.1.0, ensure that *nifi.registry.db.directory* points to
> the location of the existing H2 database to ensure it is migrated to the
> new database. Please consult the Database Properties
> <https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#database-properties>
> section of the Admin Guide for further details of these properties."
>
>
>
>
>
> On Wed, Jul 8, 2020 at 3:22 PM Rosso, Roland <
> Roland.Rosso@adventhealth.com> wrote:
>
> Thank you Bryan. Missed this link.
>
> Would you have a recommended path to upgrade from 0.1.0 keeping H2?
>
> Thank you
>
> Roland
>
>
>
> *From:* Bryan Bende <bb...@gmail.com>
> *Sent:* Wednesday, July 8, 2020 3:10 PM
> *To:* users@nifi.apache.org
> *Subject:* [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
>
>
>
> Hello,
>
>
>
> Older releases can always be found in the apache archive:
>
>
>
> https://archive.apache.org/dist/nifi/nifi-registry/
>
>
>
> Thanks,
>
>
>
> Bryan
>
>
>
> On Wed, Jul 8, 2020 at 2:14 PM Rosso, Roland <
> Roland.Rosso@adventhealth.com> wrote:
>
> Hi all,
>
> We are upgrading NiFi from 1.9.2 to 1.11.4. Our 1.9.2 uses NiFi Registry
> 0.1.0 still and we have some catching up to do.
>
> After reading the migration guidance for NiFi registry, I thought the
> prudent approach would be to upgrade to 0.2.0, then up from there.
>
> I cannot find the 0.2.0 install binaries; I found the source code but
> would rather not compile it.
>
> Could you tell me if there is a safe path from Registry 0.1.0 to 0.5.0 (or
> even 0.6.0)? We would likely keep H2 initially and secure it.
>
> We’re on Cloudera CDH 6.2.
>
>
>
> Thank you
>
> Roland Rosso
>
>
>
> This message (including any attachments) is intended only for the use of
> the individual or entity to which it is addressed and may contain
> information that is non-public, proprietary, privileged, confidential, and
> exempt from disclosure under applicable law or may constitute as attorney
> work product. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, distribution, or copying of this
> communication is strictly prohibited. If you have received this
> communication in error, notify us immediately by telephone and (i) destroy
> this message if a facsimile or (ii) delete this message immediately if this
> is an electronic communication. Thank you.
>
> This message (including any attachments) is intended only for the use of
> the individual or entity to which it is addressed and may contain
> information that is non-public, proprietary, privileged, confidential, and
> exempt from disclosure under applicable law or may constitute as attorney
> work product. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, distribution, or copying of this
> communication is strictly prohibited. If you have received this
> communication in error, notify us immediately by telephone and (i) destroy
> this message if a facsimile or (ii) delete this message immediately if this
> is an electronic communication. Thank you.
>
> This message (including any attachments) is intended only for the use of
> the individual or entity to which it is addressed and may contain
> information that is non-public, proprietary, privileged, confidential, and
> exempt from disclosure under applicable law or may constitute as attorney
> work product. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, distribution, or copying of this
> communication is strictly prohibited. If you have received this
> communication in error, notify us immediately by telephone and (i) destroy
> this message if a facsimile or (ii) delete this message immediately if this
> is an electronic communication. Thank you.
>
RE: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
Posted by "Rosso, Roland" <Ro...@AdventHealth.com>.
Hi Bryan,
I was able to go from 0.1.0 to 0.6.0. I think the database upgraded fine but the ‘new’ H2 database nifi-registry-primary.mv.db is only about 1/10 the size of the old one.
I also had to copy the flow_storage directory from 0.1.0 to 0.6.0 (did a test upgrade in parallel), but it all appears to work fine after I did that. DB size still has me concerned though.
I may have fumbled something in the steps and if you can point it out, that’d be great.
Thank you.
Roland
From: Bryan Bende <bb...@gmail.com>
Sent: Wednesday, July 8, 2020 4:04 PM
To: users@nifi.apache.org
Subject: Re: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
I'm pretty sure you can go directly from 0.1.0 to 0.6.0, there is no requirement to switch from H2 to Postgres, that was just an example in the migration for people that wanted it.
The change that was relevant was between 0.1.0 and 0.2.0, the database properties were refactored, and even if you are using H2, it will migrate from the old H2 to a new one, so just make sure you follow that guidance...
"The 0.2.0 release introduces new database configuration in nifi-registry.properties which provides more flexible options. When upgrading from 0.1.0, ensure that nifi.registry.db.directory points to the location of the existing H2 database to ensure it is migrated to the new database. Please consult the Database Properties<https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#database-properties> section of the Admin Guide for further details of these properties."
On Wed, Jul 8, 2020 at 3:22 PM Rosso, Roland <Ro...@adventhealth.com>> wrote:
Thank you Bryan. Missed this link.
Would you have a recommended path to upgrade from 0.1.0 keeping H2?
Thank you
Roland
From: Bryan Bende <bb...@gmail.com>>
Sent: Wednesday, July 8, 2020 3:10 PM
To: users@nifi.apache.org<ma...@nifi.apache.org>
Subject: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
Hello,
Older releases can always be found in the apache archive:
https://archive.apache.org/dist/nifi/nifi-registry/
Thanks,
Bryan
On Wed, Jul 8, 2020 at 2:14 PM Rosso, Roland <Ro...@adventhealth.com>> wrote:
Hi all,
We are upgrading NiFi from 1.9.2 to 1.11.4. Our 1.9.2 uses NiFi Registry 0.1.0 still and we have some catching up to do.
After reading the migration guidance for NiFi registry, I thought the prudent approach would be to upgrade to 0.2.0, then up from there.
I cannot find the 0.2.0 install binaries; I found the source code but would rather not compile it.
Could you tell me if there is a safe path from Registry 0.1.0 to 0.5.0 (or even 0.6.0)? We would likely keep H2 initially and secure it.
We’re on Cloudera CDH 6.2.
Thank you
Roland Rosso
This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.
This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.
This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.
Re: [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
Posted by Bryan Bende <bb...@gmail.com>.
I'm pretty sure you can go directly from 0.1.0 to 0.6.0, there is no
requirement to switch from H2 to Postgres, that was just an example in the
migration for people that wanted it.
The change that was relevant was between 0.1.0 and 0.2.0, the database
properties were refactored, and even if you are using H2, it will migrate
from the old H2 to a new one, so just make sure you follow that guidance...
"The 0.2.0 release introduces new database configuration in
*nifi-registry.properties* which provides more flexible options. When
upgrading from 0.1.0, ensure that *nifi.registry.db.directory* points to
the location of the existing H2 database to ensure it is migrated to the
new database. Please consult the Database Properties
<https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#database-properties>
section of the Admin Guide for further details of these properties."
On Wed, Jul 8, 2020 at 3:22 PM Rosso, Roland <Ro...@adventhealth.com>
wrote:
> Thank you Bryan. Missed this link.
>
> Would you have a recommended path to upgrade from 0.1.0 keeping H2?
>
> Thank you
>
> Roland
>
>
>
> *From:* Bryan Bende <bb...@gmail.com>
> *Sent:* Wednesday, July 8, 2020 3:10 PM
> *To:* users@nifi.apache.org
> *Subject:* [EXTERNAL] Re: NiFi registry safe upgrade path from 0.1.0
>
>
>
> Hello,
>
>
>
> Older releases can always be found in the apache archive:
>
>
>
> https://archive.apache.org/dist/nifi/nifi-registry/
>
>
>
> Thanks,
>
>
>
> Bryan
>
>
>
> On Wed, Jul 8, 2020 at 2:14 PM Rosso, Roland <
> Roland.Rosso@adventhealth.com> wrote:
>
> Hi all,
>
> We are upgrading NiFi from 1.9.2 to 1.11.4. Our 1.9.2 uses NiFi Registry
> 0.1.0 still and we have some catching up to do.
>
> After reading the migration guidance for NiFi registry, I thought the
> prudent approach would be to upgrade to 0.2.0, then up from there.
>
> I cannot find the 0.2.0 install binaries; I found the source code but
> would rather not compile it.
>
> Could you tell me if there is a safe path from Registry 0.1.0 to 0.5.0 (or
> even 0.6.0)? We would likely keep H2 initially and secure it.
>
> We’re on Cloudera CDH 6.2.
>
>
>
> Thank you
>
> Roland Rosso
>
>
>
> This message (including any attachments) is intended only for the use of
> the individual or entity to which it is addressed and may contain
> information that is non-public, proprietary, privileged, confidential, and
> exempt from disclosure under applicable law or may constitute as attorney
> work product. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, distribution, or copying of this
> communication is strictly prohibited. If you have received this
> communication in error, notify us immediately by telephone and (i) destroy
> this message if a facsimile or (ii) delete this message immediately if this
> is an electronic communication. Thank you.
>
> This message (including any attachments) is intended only for the use of
> the individual or entity to which it is addressed and may contain
> information that is non-public, proprietary, privileged, confidential, and
> exempt from disclosure under applicable law or may constitute as attorney
> work product. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, distribution, or copying of this
> communication is strictly prohibited. If you have received this
> communication in error, notify us immediately by telephone and (i) destroy
> this message if a facsimile or (ii) delete this message immediately if this
> is an electronic communication. Thank you.
>