You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Jagrut Sharma <ja...@gmail.com> on 2018/06/23 18:43:33 UTC
Registry H2 to PostgreSQL migration
Hi - I performed migration from H2 to PostgreSQL, and ran into a potential
bug.
After switching the database config, registry started up fine. But the
tables were all empty. On digging a bit deeper, I found that the default H2
database file is named nifi-registry-primary.mv.db. The migration code is
looking for a file named nifi-registry.mv.db and hence unable to find it.
The log reports 'Did not find legacy database file...'
To verify this, I renamed existing H2 file to nifi-registry.mv.db. Upon
startup, the file was detected and migration happened properly.
I can file a JIRA, but just wanted to make sure I was not missing any
config step.
Thanks.
--
Jagrut
Re: Registry H2 to PostgreSQL migration
Posted by Bryan Bende <bb...@gmail.com>.
Jagrut,
I believe you should have access to edit the NiFi Registry wiki now.
Let us know if it doesn't work.
Thanks,
Bryan
On Sat, Jun 23, 2018 at 8:28 PM, Jagrut Sharma <ja...@gmail.com> wrote:
> Thanks! My confluence username is jagrutsharma.
>
> --
> Jagrut
>
> On Sat, Jun 23, 2018 at 4:29 PM, Bryan Bende <bb...@gmail.com> wrote:
>
>> Sure, that would be good. I think you may need permissions to edit the wiki
>> page, but if you let us know your confluence account we can get you added.
>> I’m not at a computer right now, but can take care of it tomorrow if no one
>> else gets to it first.
>>
>> Thanks,
>>
>> Bryan
>>
>> On Sat, Jun 23, 2018 at 6:56 PM Jagrut Sharma <ja...@gmail.com>
>> wrote:
>>
>> > Thanks Bryan. Your explanation makes sense. Would it be ok if I add a
>> short
>> > section to this guide
>> > https://cwiki.apache.org/confluence/display/NIFIREG/Migration+Guidance
>> > with
>> > steps to switch 0.2.0 (H2) to 0.2.0 (PostgreSQL)?
>> >
>> > Thanks.
>> > --
>> > Jagrut
>> >
>> > On Sat, Jun 23, 2018 at 2:52 PM, Bryan Bende <bb...@gmail.com> wrote:
>> >
>> > > Hello,
>> > >
>> > > This is actually expected behavior...
>> > >
>> > > The migration is only setup to migrate the database from 0.1.0 which
>> was
>> > > named nifi-registry.mv.db.
>> > >
>> > > If you have an H2 DB named nifi-registry-primary.mv.db this is the name
>> > of
>> > > the new H2 DB in 0.2.0, so it’s not looking for this because the idea
>> was
>> > > you’d migrate to Postgres when you did the upgrade to 0.2.0 before you
>> > ever
>> > > had this file. Since you didn’t migrate to Postgres on the upgrade, it
>> > > migrated to a new H2 DB.
>> > >
>> > > It is nice to know that you can do it later on by renaming the file,
>> and
>> > > maybe we can add that to the migration notes wiki or admin guide.
>> > >
>> > > Thanks,
>> > >
>> > > Bryan
>> > >
>> > > On Sat, Jun 23, 2018 at 5:02 PM Kevin Doran <kd...@apache.org> wrote:
>> > >
>> > > > Hi Jagrut,
>> > > >
>> > > > I think this is configurable. How did you configure you
>> > > > nifi-registry.properties file?
>> > > >
>> > > > Thanks,
>> > > > Kevin
>> > > >
>> > > > ________________________________
>> > > > From: Jagrut Sharma <ja...@gmail.com>
>> > > > Sent: Saturday, June 23, 2018 2:43:33 PM
>> > > > To: dev@nifi.apache.org
>> > > > Subject: Registry H2 to PostgreSQL migration
>> > > >
>> > > > Hi - I performed migration from H2 to PostgreSQL, and ran into a
>> > > potential
>> > > > bug.
>> > > >
>> > > > After switching the database config, registry started up fine. But
>> the
>> > > > tables were all empty. On digging a bit deeper, I found that the
>> > default
>> > > H2
>> > > > database file is named nifi-registry-primary.mv.db. The migration
>> code
>> > is
>> > > > looking for a file named nifi-registry.mv.db and hence unable to find
>> > it.
>> > > > The log reports 'Did not find legacy database file...'
>> > > >
>> > > > To verify this, I renamed existing H2 file to nifi-registry.mv.db.
>> Upon
>> > > > startup, the file was detected and migration happened properly.
>> > > >
>> > > > I can file a JIRA, but just wanted to make sure I was not missing any
>> > > > config step.
>> > > >
>> > > > Thanks.
>> > > > --
>> > > > Jagrut
>> > > >
>> > > --
>> > > Sent from Gmail Mobile
>> > >
>> >
>> >
>> >
>> > --
>> > Jagrut
>> >
>> --
>> Sent from Gmail Mobile
>>
>
>
>
> --
> Jagrut
Re: Registry H2 to PostgreSQL migration
Posted by Jagrut Sharma <ja...@gmail.com>.
Thanks! My confluence username is jagrutsharma.
--
Jagrut
On Sat, Jun 23, 2018 at 4:29 PM, Bryan Bende <bb...@gmail.com> wrote:
> Sure, that would be good. I think you may need permissions to edit the wiki
> page, but if you let us know your confluence account we can get you added.
> I’m not at a computer right now, but can take care of it tomorrow if no one
> else gets to it first.
>
> Thanks,
>
> Bryan
>
> On Sat, Jun 23, 2018 at 6:56 PM Jagrut Sharma <ja...@gmail.com>
> wrote:
>
> > Thanks Bryan. Your explanation makes sense. Would it be ok if I add a
> short
> > section to this guide
> > https://cwiki.apache.org/confluence/display/NIFIREG/Migration+Guidance
> > with
> > steps to switch 0.2.0 (H2) to 0.2.0 (PostgreSQL)?
> >
> > Thanks.
> > --
> > Jagrut
> >
> > On Sat, Jun 23, 2018 at 2:52 PM, Bryan Bende <bb...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > This is actually expected behavior...
> > >
> > > The migration is only setup to migrate the database from 0.1.0 which
> was
> > > named nifi-registry.mv.db.
> > >
> > > If you have an H2 DB named nifi-registry-primary.mv.db this is the name
> > of
> > > the new H2 DB in 0.2.0, so it’s not looking for this because the idea
> was
> > > you’d migrate to Postgres when you did the upgrade to 0.2.0 before you
> > ever
> > > had this file. Since you didn’t migrate to Postgres on the upgrade, it
> > > migrated to a new H2 DB.
> > >
> > > It is nice to know that you can do it later on by renaming the file,
> and
> > > maybe we can add that to the migration notes wiki or admin guide.
> > >
> > > Thanks,
> > >
> > > Bryan
> > >
> > > On Sat, Jun 23, 2018 at 5:02 PM Kevin Doran <kd...@apache.org> wrote:
> > >
> > > > Hi Jagrut,
> > > >
> > > > I think this is configurable. How did you configure you
> > > > nifi-registry.properties file?
> > > >
> > > > Thanks,
> > > > Kevin
> > > >
> > > > ________________________________
> > > > From: Jagrut Sharma <ja...@gmail.com>
> > > > Sent: Saturday, June 23, 2018 2:43:33 PM
> > > > To: dev@nifi.apache.org
> > > > Subject: Registry H2 to PostgreSQL migration
> > > >
> > > > Hi - I performed migration from H2 to PostgreSQL, and ran into a
> > > potential
> > > > bug.
> > > >
> > > > After switching the database config, registry started up fine. But
> the
> > > > tables were all empty. On digging a bit deeper, I found that the
> > default
> > > H2
> > > > database file is named nifi-registry-primary.mv.db. The migration
> code
> > is
> > > > looking for a file named nifi-registry.mv.db and hence unable to find
> > it.
> > > > The log reports 'Did not find legacy database file...'
> > > >
> > > > To verify this, I renamed existing H2 file to nifi-registry.mv.db.
> Upon
> > > > startup, the file was detected and migration happened properly.
> > > >
> > > > I can file a JIRA, but just wanted to make sure I was not missing any
> > > > config step.
> > > >
> > > > Thanks.
> > > > --
> > > > Jagrut
> > > >
> > > --
> > > Sent from Gmail Mobile
> > >
> >
> >
> >
> > --
> > Jagrut
> >
> --
> Sent from Gmail Mobile
>
--
Jagrut
Re: Registry H2 to PostgreSQL migration
Posted by Bryan Bende <bb...@gmail.com>.
Sure, that would be good. I think you may need permissions to edit the wiki
page, but if you let us know your confluence account we can get you added.
I’m not at a computer right now, but can take care of it tomorrow if no one
else gets to it first.
Thanks,
Bryan
On Sat, Jun 23, 2018 at 6:56 PM Jagrut Sharma <ja...@gmail.com>
wrote:
> Thanks Bryan. Your explanation makes sense. Would it be ok if I add a short
> section to this guide
> https://cwiki.apache.org/confluence/display/NIFIREG/Migration+Guidance
> with
> steps to switch 0.2.0 (H2) to 0.2.0 (PostgreSQL)?
>
> Thanks.
> --
> Jagrut
>
> On Sat, Jun 23, 2018 at 2:52 PM, Bryan Bende <bb...@gmail.com> wrote:
>
> > Hello,
> >
> > This is actually expected behavior...
> >
> > The migration is only setup to migrate the database from 0.1.0 which was
> > named nifi-registry.mv.db.
> >
> > If you have an H2 DB named nifi-registry-primary.mv.db this is the name
> of
> > the new H2 DB in 0.2.0, so it’s not looking for this because the idea was
> > you’d migrate to Postgres when you did the upgrade to 0.2.0 before you
> ever
> > had this file. Since you didn’t migrate to Postgres on the upgrade, it
> > migrated to a new H2 DB.
> >
> > It is nice to know that you can do it later on by renaming the file, and
> > maybe we can add that to the migration notes wiki or admin guide.
> >
> > Thanks,
> >
> > Bryan
> >
> > On Sat, Jun 23, 2018 at 5:02 PM Kevin Doran <kd...@apache.org> wrote:
> >
> > > Hi Jagrut,
> > >
> > > I think this is configurable. How did you configure you
> > > nifi-registry.properties file?
> > >
> > > Thanks,
> > > Kevin
> > >
> > > ________________________________
> > > From: Jagrut Sharma <ja...@gmail.com>
> > > Sent: Saturday, June 23, 2018 2:43:33 PM
> > > To: dev@nifi.apache.org
> > > Subject: Registry H2 to PostgreSQL migration
> > >
> > > Hi - I performed migration from H2 to PostgreSQL, and ran into a
> > potential
> > > bug.
> > >
> > > After switching the database config, registry started up fine. But the
> > > tables were all empty. On digging a bit deeper, I found that the
> default
> > H2
> > > database file is named nifi-registry-primary.mv.db. The migration code
> is
> > > looking for a file named nifi-registry.mv.db and hence unable to find
> it.
> > > The log reports 'Did not find legacy database file...'
> > >
> > > To verify this, I renamed existing H2 file to nifi-registry.mv.db. Upon
> > > startup, the file was detected and migration happened properly.
> > >
> > > I can file a JIRA, but just wanted to make sure I was not missing any
> > > config step.
> > >
> > > Thanks.
> > > --
> > > Jagrut
> > >
> > --
> > Sent from Gmail Mobile
> >
>
>
>
> --
> Jagrut
>
--
Sent from Gmail Mobile
Re: Registry H2 to PostgreSQL migration
Posted by Jagrut Sharma <ja...@gmail.com>.
Thanks Bryan. Your explanation makes sense. Would it be ok if I add a short
section to this guide
https://cwiki.apache.org/confluence/display/NIFIREG/Migration+Guidance with
steps to switch 0.2.0 (H2) to 0.2.0 (PostgreSQL)?
Thanks.
--
Jagrut
On Sat, Jun 23, 2018 at 2:52 PM, Bryan Bende <bb...@gmail.com> wrote:
> Hello,
>
> This is actually expected behavior...
>
> The migration is only setup to migrate the database from 0.1.0 which was
> named nifi-registry.mv.db.
>
> If you have an H2 DB named nifi-registry-primary.mv.db this is the name of
> the new H2 DB in 0.2.0, so it’s not looking for this because the idea was
> you’d migrate to Postgres when you did the upgrade to 0.2.0 before you ever
> had this file. Since you didn’t migrate to Postgres on the upgrade, it
> migrated to a new H2 DB.
>
> It is nice to know that you can do it later on by renaming the file, and
> maybe we can add that to the migration notes wiki or admin guide.
>
> Thanks,
>
> Bryan
>
> On Sat, Jun 23, 2018 at 5:02 PM Kevin Doran <kd...@apache.org> wrote:
>
> > Hi Jagrut,
> >
> > I think this is configurable. How did you configure you
> > nifi-registry.properties file?
> >
> > Thanks,
> > Kevin
> >
> > ________________________________
> > From: Jagrut Sharma <ja...@gmail.com>
> > Sent: Saturday, June 23, 2018 2:43:33 PM
> > To: dev@nifi.apache.org
> > Subject: Registry H2 to PostgreSQL migration
> >
> > Hi - I performed migration from H2 to PostgreSQL, and ran into a
> potential
> > bug.
> >
> > After switching the database config, registry started up fine. But the
> > tables were all empty. On digging a bit deeper, I found that the default
> H2
> > database file is named nifi-registry-primary.mv.db. The migration code is
> > looking for a file named nifi-registry.mv.db and hence unable to find it.
> > The log reports 'Did not find legacy database file...'
> >
> > To verify this, I renamed existing H2 file to nifi-registry.mv.db. Upon
> > startup, the file was detected and migration happened properly.
> >
> > I can file a JIRA, but just wanted to make sure I was not missing any
> > config step.
> >
> > Thanks.
> > --
> > Jagrut
> >
> --
> Sent from Gmail Mobile
>
--
Jagrut
Re: Registry H2 to PostgreSQL migration
Posted by Bryan Bende <bb...@gmail.com>.
Hello,
This is actually expected behavior...
The migration is only setup to migrate the database from 0.1.0 which was
named nifi-registry.mv.db.
If you have an H2 DB named nifi-registry-primary.mv.db this is the name of
the new H2 DB in 0.2.0, so it’s not looking for this because the idea was
you’d migrate to Postgres when you did the upgrade to 0.2.0 before you ever
had this file. Since you didn’t migrate to Postgres on the upgrade, it
migrated to a new H2 DB.
It is nice to know that you can do it later on by renaming the file, and
maybe we can add that to the migration notes wiki or admin guide.
Thanks,
Bryan
On Sat, Jun 23, 2018 at 5:02 PM Kevin Doran <kd...@apache.org> wrote:
> Hi Jagrut,
>
> I think this is configurable. How did you configure you
> nifi-registry.properties file?
>
> Thanks,
> Kevin
>
> ________________________________
> From: Jagrut Sharma <ja...@gmail.com>
> Sent: Saturday, June 23, 2018 2:43:33 PM
> To: dev@nifi.apache.org
> Subject: Registry H2 to PostgreSQL migration
>
> Hi - I performed migration from H2 to PostgreSQL, and ran into a potential
> bug.
>
> After switching the database config, registry started up fine. But the
> tables were all empty. On digging a bit deeper, I found that the default H2
> database file is named nifi-registry-primary.mv.db. The migration code is
> looking for a file named nifi-registry.mv.db and hence unable to find it.
> The log reports 'Did not find legacy database file...'
>
> To verify this, I renamed existing H2 file to nifi-registry.mv.db. Upon
> startup, the file was detected and migration happened properly.
>
> I can file a JIRA, but just wanted to make sure I was not missing any
> config step.
>
> Thanks.
> --
> Jagrut
>
--
Sent from Gmail Mobile
Re: Registry H2 to PostgreSQL migration
Posted by Jagrut Sharma <ja...@gmail.com>.
Hi Kevin - I set the below in nifi-registry.properties file to make the
switch. Kept the H2 properties commented for reference as they came out of
the box.
:
:
# legacy database properties, used to migrate data from original DB to new
DB below
# NOTE: Users upgrading from 0.1.0 should leave these populated, but new
installs after 0.1.0 should leave these empty
nifi.registry.db.directory=./database
nifi.registry.db.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
#nifi.registry.db.directory=
#nifi.registry.db.url.append=
# database properties
#nifi.registry.db.url=jdbc:h2:./database/nifi-registry-primary;AUTOCOMMIT=OFF;DB_CLOSE_ON_EXIT=FALSE;LOCK_MODE=3;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
#nifi.registry.db.driver.class=org.h2.Driver
#nifi.registry.db.driver.directory=
#nifi.registry.db.username=nifireg
#nifi.registry.db.password=nifireg
#nifi.registry.db.maxConnections=5
#nifi.registry.db.sql.debug=false
# added by Jagrut for postgres
nifi.registry.db.url=jdbc:postgresql://localhost/nifiregistry
nifi.registry.db.driver.class=org.postgresql.Driver
nifi.registry.db.driver.directory=/Library/PostgreSQL/pgJDBC
nifi.registry.db.username=postgres
nifi.registry.db.password=postgres
nifi.registry.db.maxConnections=5
nifi.registry.db.sql.debug=false
:
:
Thanks.
--
Jagrut
On Sat, Jun 23, 2018 at 2:02 PM, Kevin Doran <kd...@apache.org> wrote:
> Hi Jagrut,
>
> I think this is configurable. How did you configure you
> nifi-registry.properties file?
>
> Thanks,
> Kevin
>
> ________________________________
> From: Jagrut Sharma <ja...@gmail.com>
> Sent: Saturday, June 23, 2018 2:43:33 PM
> To: dev@nifi.apache.org
> Subject: Registry H2 to PostgreSQL migration
>
> Hi - I performed migration from H2 to PostgreSQL, and ran into a potential
> bug.
>
> After switching the database config, registry started up fine. But the
> tables were all empty. On digging a bit deeper, I found that the default H2
> database file is named nifi-registry-primary.mv.db. The migration code is
> looking for a file named nifi-registry.mv.db and hence unable to find it.
> The log reports 'Did not find legacy database file...'
>
> To verify this, I renamed existing H2 file to nifi-registry.mv.db. Upon
> startup, the file was detected and migration happened properly.
>
> I can file a JIRA, but just wanted to make sure I was not missing any
> config step.
>
> Thanks.
> --
> Jagrut
>
--
Jagrut
Re: Registry H2 to PostgreSQL migration
Posted by Kevin Doran <kd...@apache.org>.
Hi Jagrut,
I think this is configurable. How did you configure you nifi-registry.properties file?
Thanks,
Kevin
________________________________
From: Jagrut Sharma <ja...@gmail.com>
Sent: Saturday, June 23, 2018 2:43:33 PM
To: dev@nifi.apache.org
Subject: Registry H2 to PostgreSQL migration
Hi - I performed migration from H2 to PostgreSQL, and ran into a potential
bug.
After switching the database config, registry started up fine. But the
tables were all empty. On digging a bit deeper, I found that the default H2
database file is named nifi-registry-primary.mv.db. The migration code is
looking for a file named nifi-registry.mv.db and hence unable to find it.
The log reports 'Did not find legacy database file...'
To verify this, I renamed existing H2 file to nifi-registry.mv.db. Upon
startup, the file was detected and migration happened properly.
I can file a JIRA, but just wanted to make sure I was not missing any
config step.
Thanks.
--
Jagrut