You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@fineract.apache.org by Isaac Kamga <is...@mifos.org> on 2019/05/02 02:32:08 UTC
Re: updates On Migration from MariaDB to PostgreSQL
Hello Juhan,
I've updated the PRs with commits to fix the errors you were experiencing.
Could you kindly wipe Cassandra and PostgreSQL clean, test once more and
provide feedback on how it goes ?
Cheers,
Isaac Kamga.
On Tue, Apr 9, 2019 at 4:36 PM Isaac Kamga <is...@mifos.org> wrote:
> Hey Juhan,
>
> Thanks for testing.
>
> Wipe clean your Cassandra keyspaces first.
>
> I suspect that what's interferring is the keyspaces from your last run
> when using MariaDB.
>
> Please provide feedback how it goes.
>
> Cheers,
> Isaac Kamga.
>
> On Sun, Mar 31, 2019 at 10:00 AM Juhan Aasaru <aa...@gmail.com> wrote:
>
>> I Isaac,
>>
>> I pulled the changes and rebuilt the projects.
>>
>> Now when I run (postgres and cassandra working locally):
>>
>> java -Ddemoserver.persistent=true -Ddemoserver.provision=true
>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>
>> I get an error here:
>>
>> 11:18:42.066 [main] INFO o.s.b.c.e.j.JettyEmbeddedServletContainer -
>> Jetty
>> started on port(s) 2024 (http/1.1)
>> 11:18:42.067 [main] INFO o.s.c.n.e.EurekaDiscoveryClientConfiguration -
>> Updating port to 2024
>> 11:18:42.072 [main] INFO o.a.f.c.customer.CustomerApplication - Started
>> CustomerApplication in 16.423 seconds (JVM running for 17.342)
>> 11:18:57.435 [qtp1479696465-21] INFO o.a.f.c.l.c.ServiceExceptionFilter -
>> Responding with a service error ServiceError{code=409,
>> message='Application
>> identity-v1 already exists!'}
>> 11:18:57.464 [Thread-16] INFO
>> o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing
>>
>> org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@4d50efb8
>> :
>> startup date [Sun Mar 31 11:12:52 EEST 2019]; parent:
>>
>> org.springframework.context.annotation.AnnotationConfigApplicationContext@75a1cd57
>>
>> I understand I have to use -Ddemoserver.provision=true flag to provision
>> the initial database so I won't be able to take that one out.
>>
>> Juhan
>>
>>
>> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval R, 29.
>> märts 2019 kell 04:17:
>>
>> > Hey Juhan,
>> >
>> > I have updated the Pull request to fineract-cn-provisioner with a commit
>> > [1] which fixes the errors you're experiencing.
>> >
>> > I'll advise that you test using option -Ddemoserver.persistent=true and
>> > -Ddemoserver.lite=true and we're testing just the minimal set of
>> services
>> > now. In fact,I tested and the tail of my logs showed thus
>> > *identity-v1 address:http://localhost:2021/identity/v1
>> > <
>> >
>> https://www.google.com/url?q=http://localhost:2021/identity/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNEjCID96mw3LF1ZIPmjEIQkHh_LcQ
>> > >*
>> >
>> > *office-v1 address:http://localhost:2023/office/v1
>> > <
>> >
>> https://www.google.com/url?q=http://localhost:2023/office/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNGjyk1BHroLddOTd0FhJUJG4zdVag
>> > >customer-v1
>> > address:http://localhost:2024/customer/v1
>> > <
>> >
>> https://www.google.com/url?q=http://localhost:2024/customer/v1&sa=D&source=hangouts&ust=1553910362599000&usg=AFQjCNGfcoJZIxmcsX34YeEXRUzS0oC2hA
>> > >*
>> > Albeit, this error [2] turned up related to the Notifications service.
>> Just
>> > ignore it for now as it will be handled down the line during extensive
>> > integration testing.
>> >
>> > We can keep testing using -Ddemoserver.provision=true for later too when
>> > we've migrated and tested the individual services.
>> >
>> > Can you kindly update your provisioner repository and test once more and
>> > let's keep fixing ?
>> >
>> > [1]
>> >
>> >
>> https://github.com/Izakey/fineract-cn-provisioner/commit/b83331b723f175003ccf069764414e4ab371662c
>> >
>> > [2] https://pastebin.com/DBzNyHir
>> >
>> > Cheers,
>> > Isaac Kamga.
>> >
>> > On Wed, Mar 27, 2019 at 10:23 PM Isaac Kamga <is...@mifos.org>
>> > wrote:
>> >
>> > > Hello Juhan,
>> > >
>> > > Thank you for taking the time to do some testing.
>> > >
>> > > On Wed, Mar 27, 2019 at 10:56 AM Juhan Aasaru <aa...@gmail.com>
>> wrote:
>> > >
>> > >> Hi Isaac,
>> > >>
>> > >> great initiative towards pushing to PostgreSQL.
>> > >>
>> > >> I looked at your instructions and tried to run your changes locally.
>> > >> The sh script came in handy to quickly clone and build everything
>> that
>> > was
>> > >> needed.
>> > >> But then I got confused about provisioning.
>> > >>
>> > >> Question #1: How exactly I should run demo-server? I tried the
>> following
>> > >> options:
>> > >>
>> > >> 1. I didn't have postgre and cassandra running locally.
>> > >> java -jar -Ddemoserver.lite=true
>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>> > >>
>> > >> resulted
>> > >>
>> > >> org.springframework.beans.factory.BeanCreationException: Error
>> creating
>> > >> bean with name 'provisionerInitializer':
>> > >> Invocation of init method failed; nested exception is
>> > >> java.lang.IllegalStateException: Could not initialize service!
>> > >> Caused by: org.postgresql.util.PSQLException: FATAL: database
>> "seshat"
>> > >> does
>> > >> not exist
>> > >>
>> > >
>> > > I haven't done testing using this option yet.
>> > >
>> > >
>> > >>
>> > >> 2. I started posgresql (and created a database seshat) and cassandra
>> and
>> > >> ran:
>> > >> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>> > >> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 \
>> > >> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>> > >> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>> > >>
>> > >> Initially applications started up one after anther
>> > >> (ProvisionerApplication,
>> > >> RhythmApplication, OfficeApplication)
>> > >> but CustomerApplication threw an exception:
>> > >>
>> > >> 11:50:39.377 [qtp1424482154-21] WARN
>> o.e.jetty.servlet.ServletHandler -
>> > >> org.springframework.web.util.NestedServletException: Request
>> processing
>> > >> failed; nested exception is java.lang.IllegalStateException: Could
>> not
>> > >> insert org.apache.fineract.cn.provisioner.tenant info!
>> > >> Caused by: org.postgresql.util.PSQLException: ERROR: relation
>> > >> "seshat.tenants" does not exist
>> > >>
>> > >
>> > > Try running the same command above without the provision=true option
>> and
>> > > share what you get.
>> > >
>> > > I had to create database seshat in postgresql and the can't find
>> database
>> > > error went away. Down the road, also encountered the same error
>> > > <https://pastebin.com/qCT1T5KM> "Could not insert ....
>> > provisioner.tenant
>> > > info !"
>> > > I'm currently working on resolving this.
>> > >
>> > >
>> > >>
>> > >> Question #2 - do I still need to have Mysql running? I see that you
>> > >> haven't
>> > >> made code changes to identity
>> > >> which as I understand still uses Mysql.
>> > >>
>> > >
>> > > I don't think you need MySQL running to test the migrations currently
>> > > underway and there was no need for code changes in the identity
>> service.
>> > > However, it could be helpful to see the structure of the databases in
>> > MySQL
>> > > so that it helps with debugging and testing.
>> > >
>> > > Whenever you're available, tell me so we schedule a call so I look
>> into
>> > > what your experiencing too.
>> > >
>> > > Cheers,
>> > > Isaac Kamga.
>> > >
>> > >
>> > >>
>> > >> Kind regards
>> > >> Juhan
>> > >>
>> > >>
>> > >> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval L,
>> 23.
>> > >> märts 2019 kell 04:31:
>> > >>
>> > >> > Hello fineracters,
>> > >> >
>> > >> > Trust that you had a great week.
>> > >> >
>> > >> > I just finished migrating libraries PostgreSQL[1] and Test[2] from
>> > >> MariaDB
>> > >> > to PostgreSQL, mostly adapting SQL statements & database
>> connections
>> > to
>> > >> > work with PostgreSQL and reverting back to Hibernate ORM until
>> > database
>> > >> > migrations are completed.
>> > >> >
>> > >> > I have also migrated the "minimal" services viz provisioner[3],
>> > >> rhythm[4],
>> > >> > office[5], template[6] and customer[7] from MariaDB to PostgreSQL.
>> The
>> > >> > Spring datasource credentials had to be incorporated into
>> > >> application.yaml
>> > >> > files to make things work too.
>> > >> >
>> > >> > Given I updated service-starter [8] and demo-server [9], developers
>> > can
>> > >> > test the migrated services when running the
>> *-Ddemoserver.lite=true*
>> > >> option
>> > >> > which builds the minimal set of services.
>> > >> >
>> > >> > You'll observe that some of the pull requests are heading to
>> develop
>> > >> > branches ( It appears comparing changes and creating PRs towards
>> the
>> > >> same
>> > >> > branch name from origin to upstream expires after a few minutes on
>> > >> > Github... weird )
>> > >> >
>> > >> > Here is a script [10] to enable anyone clone and do an
>> > integration-test
>> > >> of
>> > >> > the new changes using demo-server.
>> > >> >
>> > >> > Kindly test these changes and provide feedback.
>> > >> >
>> > >> > At Your Service,
>> > >> > Isaac Kamga.
>> > >> >
>> > >> > [1] https://github.com/apache/fineract-cn-postgresql/pull/1
>> > >> > [2] https://github.com/apache/fineract-cn-test/pull/7
>> > >> > [3] https://github.com/apache/fineract-cn-provisioner/pull/6
>> > >> > [4] https://github.com/apache/fineract-cn-rhythm/pull/8
>> > >> > [5] https://github.com/apache/fineract-cn-office/pull/9
>> > >> > [6] https://github.com/apache/fineract-cn-template/pull/6
>> > >> > [7] https://github.com/apache/fineract-cn-customer/pull/10
>> > >> > [8] https://github.com/apache/fineract-cn-service-starter/pull/8
>> > >> > [9] https://github.com/apache/fineract-cn-demo-server/pull/27
>> > >> > [10]
>> > >> >
>> > >> >
>> > >>
>> >
>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>> > >> >
>> > >>
>> > >
>> >
>>
>
Re: updates On Migration from MariaDB to PostgreSQL
Posted by Ebenezer Graham <eg...@alustudent.com>.
Pardon me, forgot to add the SQLExcpetion and authentication log:
Full log: https://pastebin.com/GYubSKrj
Note that the authentication works for ordinary login.
*SQL Error:*
22:37:49.509 [pool-8-thread-1] WARN o.h.e.jdbc.spi.SqlExceptionHelper -
SQL Error: 0, SQLState: 42P01
22:37:49.510 [pool-8-thread-1] ERROR o.h.e.jdbc.spi.SqlExceptionHelper -
ERROR: relation "khepri_beats" does not exist
Position: 271
*Authentication error:*
22:54:30.665 [qtp1221991240-15] WARN o.e.jetty.servlet.ServletHandler -
org.springframework.web.util.NestedServletException: Request processing
failed; nested exception is
ServiceException{serviceError=ServiceError{code=500, message='An error
occurred while attempting to authenticate a user.'}}
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:982)
Regards,
Graham
On Sat, 11 May 2019 at 22:09, Ebenezer Graham <eg...@alustudent.com>
wrote:
> Hello Devs/Isaac,
>
> I tested the service in both lite and full mode
>
> Full mode resulted in an error
> Complete log here: https://pastebin.com/J0YXvkLq
>
> Time: 968.131
> There was 1 failure:
> 1) startDevServer(org.apache.fineract.cn.dev.ServiceRunner)
> java.lang.IllegalArgumentException: Index for header 'show' is 4 but
> CSVRecord only has 4 values!
> at org.apache.commons.csv.CSVRecord.get(CSVRecord.java:110)
> at
> org.apache.fineract.cn.accounting.importer.LedgerImporter.toLedger(LedgerImporter.java:106)
>
> I suspect the error is caused during the parsing of i
> mportdata/ledger-missing-name-case.csv as it has one missing value in row
> 1.
>
> However, this is weird because, in the competent test, the test for i
> mportdata/ledger-missing-name-case.csv doesn't fail but the test for
> importdata/account-from-a-real-case.csv fails.
>
> On top of that, the parser in LedgerImporter::importCSV uses the format
> CSVFormat.RFC4180.withHeader() and so the import should work just fine.
>
> I have been debugging it for a while now but no success in resolving the
> issue. Let me know if you have had luck on your end.
>
> Secondly, I did a black box testing in lite mode and got an SQLException
> whenever I tried creating a customer or ledger and automatically logs out.
>
> Looking forward to your response.
>
> *At your service,*
>
> *Ebenezer Graham*
>
> *BSc (Hons) Computing*
>
>
> [image: EmailSignature.png]
>
> African Leadership University,
>
> Power Mill Road, Pamplemousses,
>
> Mauritius.
> GitHub <https://github.com/ebenezergraham> | LinkedIn
> <https://www.linkedin.com/in/ebenezer-graham/> | ebenezergraham.me
> *Skype*:
> ebenezer.graham <http://ebenezergraham.me/>
>
> *This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they are
> addressed. If you have received this email in error please notify the
> sender and then delete this email. If you are not the intended recipient
> you are notified that disclosing, copying, distributing or taking any
> action in reliance on the contents of this information is strictly
> prohibited.*
>
>
> On Fri, 10 May 2019 at 23:09, Isaac Kamga <is...@mifos.org> wrote:
>
>> Hello Graham,
>>
>> Thanks for the feedback.
>>
>> Given the test was successful in lite mode, did you also check that the
>> tables in databases seshat and playground in PostgreSQL were rightly
>> created ?
>>
>> As for the error you encountered, I think it's not from the PostgreSQL
>> setup since you've encountered it before in the MariaDB one.
>>
>> Kindly test with all services once more and share the entire error log
>> using pastebin.
>>
>> At Your Service,
>> Isaac Kamga.
>>
>> On Fri, May 10, 2019 at 4:06 AM Ebenezer Graham <eg...@alustudent.com>
>> wrote:
>>
>>> Hi Isaac,
>>>
>>> Thanks for this effort.
>>>
>>> I have tried testing with all the services. but, I get the error below. *complete
>>> log here: *https://pastebin.com/013aTJ8E
>>>
>>>
>>> 1. 06:21:47.050 [Thread-13] INFO
>>> o.e.j.server.handler.ContextHandler - Stopped
>>> o.s.b.c.e.j.JettyEmbeddedWebAppContext@3e1fd62b
>>> {/identity/v1,file:///tmp/jetty-docbase.1273617126957352135.2021/,UNAVAILABLE}
>>> 2. E06:21:47.790 [DefaultMessageListenerContainer-1] ERROR
>>> o.s.j.l.DefaultMessageListenerContainer - Could not refresh JMS Connection
>>> for destination 'portfolio-v1' - retrying using FixedBackOff{interval=5000,
>>> currentAttempts=0, maxAttempts=unlimited}. Cause: Error while attempting to
>>> retrieve a connection from the pool; nested exception is
>>> javax.jms.JMSException: Could not connect to broker URL:
>>> tcp://localhost:61616. Reason: java.net.ConnectException: Connection
>>> refused (Connection refused)
>>>
>>>
>>>
>>> I usually get this same error even with the MariaDB setup. I know its
>>> related to the Message broker but I don't know how to resolve it or how to
>>> avoid it. Can anyone share their experience on this issue?
>>>
>>> Also, I dropped the playground and seshat keyspaces in Cassandra as well
>>> as in Postgres.
>>> I changed the max-connection to 120 in PostgresSQL and also restarted
>>> the system.
>>>
>>> I am convinced the problem is not with PostgreSQL or Cassandra, plus, it
>>> worked a few hours ago when I started it in lite mode.
>>>
>>> PS: In light of this, how soon will this be merged into the develop
>>> branches. I would love to completely switch to PostgreSQL. Toggling
>>> between MariaDB and PostgreSQL is not easy :)
>>>
>>> *At your service,*
>>> *Graham.*
>>>
>>> On Thu, 9 May 2019 at 20:13, Isaac Kamga <is...@mifos.org> wrote:
>>>
>>>> Hello Dev,
>>>>
>>>> Thanks very much for testing Graham
>>>>
>>>> I've raised pull requests for the Migration to PostgreSQL. Demo-server
>>>> runs to completion successfully in both cases and databases and tables are
>>>> rightly being created.
>>>> I had to make some adjustments to default-setup's ledgers.csv file
>>>> because of an error that came from the CSVParser.
>>>>
>>>> *Testing demo-server lite for minimal set of services*
>>>> I have fixed the issue with connecting to the right databases. You can
>>>> use the postgres-setup-lite.sh
>>>> <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh>
>>>> script to test the minimal set of libraries and services. After the run,
>>>> you should be able to ascertain that databases are created and tables are
>>>> created where they have to be.
>>>>
>>>> +Juhan Aasaru <aa...@gmail.com> please provide feedback how this test
>>>> goes.
>>>>
>>>> *Testing demo-server for all services*
>>>> Given that the connection issues have been resolved, it's time to do
>>>> more testing;
>>>> @Juhan, @Graham and anyone else, in order to test the migration in it's
>>>> entirety using postgres-setup.sh script
>>>> <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup.sh>,
>>>> ensure to;
>>>>
>>>> 1. Install PostgreSQL
>>>> 2. Change password for user postgres to postgres
>>>> 3. Change max_connections to 120 in your postgresql.conf file ( usually
>>>> in /etc/postgresql/... ) and restart the Postgresql service.Many
>>>> connections to databases happening since all services are started.
>>>> 4. Ensure Cassandra has no keyspaces seshat and playground
>>>> 5. cd into a completely new directory from your usual Fineract CN
>>>> development pwd
>>>> 5. In integration-tests/fineract-cn-demo-server/build/libs/ and Run
>>>> demo-server jar using *java -Ddemoserver.provision=true
>>>> -Ddemoserver.persistent=true
>>>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>>> <http://127.0.0.1:9042/> -Dcassandra.cluster.user=cassandra
>>>> -Dcassandra.cluster.pwd=password -Dcustom.postgresql.host=localhost
>>>> -Dcustom.postgresql.user=postgres -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar*
>>>>
>>>> Kindly provide feedback on how the run goes in the both cases.
>>>>
>>>> At Your Service,
>>>> Isaac Kamga.
>>>>
>>>> On Fri, May 3, 2019 at 7:21 PM Ebenezer Graham <
>>>> egraham15@alustudent.com> wrote:
>>>>
>>>>> Hello Isaac / Juhan
>>>>>
>>>>> I missed a number of relevant steps and this led to the initial
>>>>> failure.
>>>>>
>>>>> The test has been successful and I will list the conditions which led
>>>>> to the successful run below.
>>>>>
>>>>> 1. I created a new directory and cloned the repositories using the
>>>>> script you provided (
>>>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>>>>> )
>>>>>
>>>>> 2. I installed PostgreSQL server and changed the postgres user's
>>>>> password to "postgres"
>>>>>
>>>>> 3. I used the command below to start the demo-server.
>>>>>
>>>>> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>>>>> -Ddemoserver.lite=true -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>>>> -Dcustom.postgresql.host=localhost -Dcustom.postgresql.user=postgres -jar
>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>
>>>>> @Juhan, please try this approach and let's know the outcome.
>>>>>
>>>>> You should get an output similar to this.
>>>>>
>>>>> 18:11:28.717 [Thread-16] INFO o.e.jetty.server.AbstractConnector -
>>>>> Stopped ServerConnector@59a94d0f{HTTP/1.1,[http/1.1]}{0.0.0.0:2020}
>>>>> 18:11:28.720 [Thread-16] INFO o.e.j.server.handler.ContextHandler -
>>>>> Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@1213ffbc
>>>>> {/provisioner/v1,file:///tmp/jetty-docbase.8613597725021372791.2020/,UNAVAILABLE}
>>>>> identity-v1 address:http://localhost:2021/identity/v1
>>>>> office-v1 address:http://localhost:2023/office/v1
>>>>> customer-v1 address:http://localhost:2024/customer/v1
>>>>>
>>>>> At your service
>>>>>
>>>>>
>>>>> On Fri, 3 May 2019 at 21:09, Ebenezer Graham <eg...@alustudent.com>
>>>>> wrote:
>>>>>
>>>>>> Hello Isaac/Juhan,
>>>>>>
>>>>>> I have tried testing the code in lite mode for 3 times and I got the error
>>>>>> below.
>>>>>> https://pastebin.com/TaJqSNjQ
>>>>>>
>>>>>> Before the failure, I got this info.
>>>>>>
>>>>>> 16:56:03.519 [qtp1262607708-22] INFO
>>>>>> o.a.f.c.l.c.ServiceExceptionFilter - Responding with a service error
>>>>>> ServiceError{code=409, message='The given identity instance didnt recognize
>>>>>> the system token as valid. Perhaps the system keys for the provisioner or
>>>>>> for the identity manager are misconfigured?'}
>>>>>> 16:57:03.620 [qtp1262607708-20] INFO
>>>>>> o.a.f.c.l.c.ServiceExceptionFilter - Responding with a service error
>>>>>> ServiceError{code=409, message='The given identity instance didnt recognize
>>>>>> the system token as valid. Perhaps the system keys for the provisioner or
>>>>>> for the identity manager are misconfigured?'}
>>>>>>
>>>>>> I will keep testing to get more insights into the error.
>>>>>>
>>>>>> *At your service,*
>>>>>>
>>>>>> *Ebenezer Graham*
>>>>>>
>>>>>> *BSc (Hons) Computing*
>>>>>>
>>>>>>
>>>>>> [image: EmailSignature.png]
>>>>>>
>>>>>> African Leadership University,
>>>>>>
>>>>>> Power Mill Road, Pamplemousses,
>>>>>>
>>>>>> Mauritius.
>>>>>> GitHub <https://github.com/ebenezergraham> | LinkedIn
>>>>>> <https://www.linkedin.com/in/ebenezer-graham/> | ebenezergraham.me
>>>>>> *Skype*:
>>>>>> ebenezer.graham <http://ebenezergraham.me/>
>>>>>>
>>>>>> *This email and any files transmitted with it are confidential and
>>>>>> intended solely for the use of the individual or entity to whom they are
>>>>>> addressed. If you have received this email in error please notify the
>>>>>> sender and then delete this email. If you are not the intended recipient
>>>>>> you are notified that disclosing, copying, distributing or taking any
>>>>>> action in reliance on the contents of this information is strictly
>>>>>> prohibited.*
>>>>>>
>>>>>>
>>>>>> On Thu, 2 May 2019 at 06:38, Isaac Kamga <is...@mifos.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello Juhan,
>>>>>>>
>>>>>>> I've updated the PRs with commits to fix the errors you were
>>>>>>> experiencing.
>>>>>>>
>>>>>>> Could you kindly wipe Cassandra and PostgreSQL clean, test once more
>>>>>>> and provide feedback on how it goes ?
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Isaac Kamga.
>>>>>>>
>>>>>>> On Tue, Apr 9, 2019 at 4:36 PM Isaac Kamga <is...@mifos.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hey Juhan,
>>>>>>>>
>>>>>>>> Thanks for testing.
>>>>>>>>
>>>>>>>> Wipe clean your Cassandra keyspaces first.
>>>>>>>>
>>>>>>>> I suspect that what's interferring is the keyspaces from your last
>>>>>>>> run when using MariaDB.
>>>>>>>>
>>>>>>>> Please provide feedback how it goes.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Isaac Kamga.
>>>>>>>>
>>>>>>>> On Sun, Mar 31, 2019 at 10:00 AM Juhan Aasaru <aa...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I Isaac,
>>>>>>>>>
>>>>>>>>> I pulled the changes and rebuilt the projects.
>>>>>>>>>
>>>>>>>>> Now when I run (postgres and cassandra working locally):
>>>>>>>>>
>>>>>>>>> java -Ddemoserver.persistent=true -Ddemoserver.provision=true
>>>>>>>>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>>>>>>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>>>>>>>> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>>>>>
>>>>>>>>> I get an error here:
>>>>>>>>>
>>>>>>>>> 11:18:42.066 [main] INFO
>>>>>>>>> o.s.b.c.e.j.JettyEmbeddedServletContainer - Jetty
>>>>>>>>> started on port(s) 2024 (http/1.1)
>>>>>>>>> 11:18:42.067 [main] INFO
>>>>>>>>> o.s.c.n.e.EurekaDiscoveryClientConfiguration -
>>>>>>>>> Updating port to 2024
>>>>>>>>> 11:18:42.072 [main] INFO o.a.f.c.customer.CustomerApplication -
>>>>>>>>> Started
>>>>>>>>> CustomerApplication in 16.423 seconds (JVM running for 17.342)
>>>>>>>>> 11:18:57.435 [qtp1479696465-21] INFO
>>>>>>>>> o.a.f.c.l.c.ServiceExceptionFilter -
>>>>>>>>> Responding with a service error ServiceError{code=409,
>>>>>>>>> message='Application
>>>>>>>>> identity-v1 already exists!'}
>>>>>>>>> 11:18:57.464 [Thread-16] INFO
>>>>>>>>> o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing
>>>>>>>>>
>>>>>>>>> org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@4d50efb8
>>>>>>>>> :
>>>>>>>>> startup date [Sun Mar 31 11:12:52 EEST 2019]; parent:
>>>>>>>>>
>>>>>>>>> org.springframework.context.annotation.AnnotationConfigApplicationContext@75a1cd57
>>>>>>>>>
>>>>>>>>> I understand I have to use -Ddemoserver.provision=true flag to
>>>>>>>>> provision
>>>>>>>>> the initial database so I won't be able to take that one out.
>>>>>>>>>
>>>>>>>>> Juhan
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval
>>>>>>>>> R, 29.
>>>>>>>>> märts 2019 kell 04:17:
>>>>>>>>>
>>>>>>>>> > Hey Juhan,
>>>>>>>>> >
>>>>>>>>> > I have updated the Pull request to fineract-cn-provisioner with
>>>>>>>>> a commit
>>>>>>>>> > [1] which fixes the errors you're experiencing.
>>>>>>>>> >
>>>>>>>>> > I'll advise that you test using option
>>>>>>>>> -Ddemoserver.persistent=true and
>>>>>>>>> > -Ddemoserver.lite=true and we're testing just the minimal set of
>>>>>>>>> services
>>>>>>>>> > now. In fact,I tested and the tail of my logs showed thus
>>>>>>>>> > *identity-v1 address:http://localhost:2021/identity/v1
>>>>>>>>> > <
>>>>>>>>> >
>>>>>>>>> https://www.google.com/url?q=http://localhost:2021/identity/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNEjCID96mw3LF1ZIPmjEIQkHh_LcQ
>>>>>>>>> > >*
>>>>>>>>> >
>>>>>>>>> > *office-v1 address:http://localhost:2023/office/v1
>>>>>>>>> > <
>>>>>>>>> >
>>>>>>>>> https://www.google.com/url?q=http://localhost:2023/office/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNGjyk1BHroLddOTd0FhJUJG4zdVag
>>>>>>>>> > >customer-v1
>>>>>>>>> > address:http://localhost:2024/customer/v1
>>>>>>>>> > <
>>>>>>>>> >
>>>>>>>>> https://www.google.com/url?q=http://localhost:2024/customer/v1&sa=D&source=hangouts&ust=1553910362599000&usg=AFQjCNGfcoJZIxmcsX34YeEXRUzS0oC2hA
>>>>>>>>> > >*
>>>>>>>>> > Albeit, this error [2] turned up related to the Notifications
>>>>>>>>> service. Just
>>>>>>>>> > ignore it for now as it will be handled down the line during
>>>>>>>>> extensive
>>>>>>>>> > integration testing.
>>>>>>>>> >
>>>>>>>>> > We can keep testing using -Ddemoserver.provision=true for later
>>>>>>>>> too when
>>>>>>>>> > we've migrated and tested the individual services.
>>>>>>>>> >
>>>>>>>>> > Can you kindly update your provisioner repository and test once
>>>>>>>>> more and
>>>>>>>>> > let's keep fixing ?
>>>>>>>>> >
>>>>>>>>> > [1]
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> https://github.com/Izakey/fineract-cn-provisioner/commit/b83331b723f175003ccf069764414e4ab371662c
>>>>>>>>> >
>>>>>>>>> > [2] https://pastebin.com/DBzNyHir
>>>>>>>>> >
>>>>>>>>> > Cheers,
>>>>>>>>> > Isaac Kamga.
>>>>>>>>> >
>>>>>>>>> > On Wed, Mar 27, 2019 at 10:23 PM Isaac Kamga <
>>>>>>>>> isaac.kamga@mifos.org>
>>>>>>>>> > wrote:
>>>>>>>>> >
>>>>>>>>> > > Hello Juhan,
>>>>>>>>> > >
>>>>>>>>> > > Thank you for taking the time to do some testing.
>>>>>>>>> > >
>>>>>>>>> > > On Wed, Mar 27, 2019 at 10:56 AM Juhan Aasaru <
>>>>>>>>> aasaru@gmail.com> wrote:
>>>>>>>>> > >
>>>>>>>>> > >> Hi Isaac,
>>>>>>>>> > >>
>>>>>>>>> > >> great initiative towards pushing to PostgreSQL.
>>>>>>>>> > >>
>>>>>>>>> > >> I looked at your instructions and tried to run your changes
>>>>>>>>> locally.
>>>>>>>>> > >> The sh script came in handy to quickly clone and build
>>>>>>>>> everything that
>>>>>>>>> > was
>>>>>>>>> > >> needed.
>>>>>>>>> > >> But then I got confused about provisioning.
>>>>>>>>> > >>
>>>>>>>>> > >> Question #1: How exactly I should run demo-server? I tried
>>>>>>>>> the following
>>>>>>>>> > >> options:
>>>>>>>>> > >>
>>>>>>>>> > >> 1. I didn't have postgre and cassandra running locally.
>>>>>>>>> > >> java -jar -Ddemoserver.lite=true
>>>>>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>>>>> > >>
>>>>>>>>> > >> resulted
>>>>>>>>> > >>
>>>>>>>>> > >> org.springframework.beans.factory.BeanCreationException:
>>>>>>>>> Error creating
>>>>>>>>> > >> bean with name 'provisionerInitializer':
>>>>>>>>> > >> Invocation of init method failed; nested exception is
>>>>>>>>> > >> java.lang.IllegalStateException: Could not initialize service!
>>>>>>>>> > >> Caused by: org.postgresql.util.PSQLException: FATAL: database
>>>>>>>>> "seshat"
>>>>>>>>> > >> does
>>>>>>>>> > >> not exist
>>>>>>>>> > >>
>>>>>>>>> > >
>>>>>>>>> > > I haven't done testing using this option yet.
>>>>>>>>> > >
>>>>>>>>> > >
>>>>>>>>> > >>
>>>>>>>>> > >> 2. I started posgresql (and created a database seshat) and
>>>>>>>>> cassandra and
>>>>>>>>> > >> ran:
>>>>>>>>> > >> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>>>>>>>>> > >> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 \
>>>>>>>>> > >> -Dcassandra.cluster.user=cassandra
>>>>>>>>> -Dcassandra.cluster.pwd=password
>>>>>>>>> > >> -Ddemoserver.lite=true -jar
>>>>>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>>>>> > >>
>>>>>>>>> > >> Initially applications started up one after anther
>>>>>>>>> > >> (ProvisionerApplication,
>>>>>>>>> > >> RhythmApplication, OfficeApplication)
>>>>>>>>> > >> but CustomerApplication threw an exception:
>>>>>>>>> > >>
>>>>>>>>> > >> 11:50:39.377 [qtp1424482154-21] WARN
>>>>>>>>> o.e.jetty.servlet.ServletHandler -
>>>>>>>>> > >> org.springframework.web.util.NestedServletException: Request
>>>>>>>>> processing
>>>>>>>>> > >> failed; nested exception is java.lang.IllegalStateException:
>>>>>>>>> Could not
>>>>>>>>> > >> insert org.apache.fineract.cn.provisioner.tenant info!
>>>>>>>>> > >> Caused by: org.postgresql.util.PSQLException: ERROR: relation
>>>>>>>>> > >> "seshat.tenants" does not exist
>>>>>>>>> > >>
>>>>>>>>> > >
>>>>>>>>> > > Try running the same command above without the provision=true
>>>>>>>>> option and
>>>>>>>>> > > share what you get.
>>>>>>>>> > >
>>>>>>>>> > > I had to create database seshat in postgresql and the can't
>>>>>>>>> find database
>>>>>>>>> > > error went away. Down the road, also encountered the same error
>>>>>>>>> > > <https://pastebin.com/qCT1T5KM> "Could not insert ....
>>>>>>>>> > provisioner.tenant
>>>>>>>>> > > info !"
>>>>>>>>> > > I'm currently working on resolving this.
>>>>>>>>> > >
>>>>>>>>> > >
>>>>>>>>> > >>
>>>>>>>>> > >> Question #2 - do I still need to have Mysql running? I see
>>>>>>>>> that you
>>>>>>>>> > >> haven't
>>>>>>>>> > >> made code changes to identity
>>>>>>>>> > >> which as I understand still uses Mysql.
>>>>>>>>> > >>
>>>>>>>>> > >
>>>>>>>>> > > I don't think you need MySQL running to test the migrations
>>>>>>>>> currently
>>>>>>>>> > > underway and there was no need for code changes in the
>>>>>>>>> identity service.
>>>>>>>>> > > However, it could be helpful to see the structure of the
>>>>>>>>> databases in
>>>>>>>>> > MySQL
>>>>>>>>> > > so that it helps with debugging and testing.
>>>>>>>>> > >
>>>>>>>>> > > Whenever you're available, tell me so we schedule a call so I
>>>>>>>>> look into
>>>>>>>>> > > what your experiencing too.
>>>>>>>>> > >
>>>>>>>>> > > Cheers,
>>>>>>>>> > > Isaac Kamga.
>>>>>>>>> > >
>>>>>>>>> > >
>>>>>>>>> > >>
>>>>>>>>> > >> Kind regards
>>>>>>>>> > >> Juhan
>>>>>>>>> > >>
>>>>>>>>> > >>
>>>>>>>>> > >> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas
>>>>>>>>> kuupäeval L, 23.
>>>>>>>>> > >> märts 2019 kell 04:31:
>>>>>>>>> > >>
>>>>>>>>> > >> > Hello fineracters,
>>>>>>>>> > >> >
>>>>>>>>> > >> > Trust that you had a great week.
>>>>>>>>> > >> >
>>>>>>>>> > >> > I just finished migrating libraries PostgreSQL[1] and
>>>>>>>>> Test[2] from
>>>>>>>>> > >> MariaDB
>>>>>>>>> > >> > to PostgreSQL, mostly adapting SQL statements & database
>>>>>>>>> connections
>>>>>>>>> > to
>>>>>>>>> > >> > work with PostgreSQL and reverting back to Hibernate ORM
>>>>>>>>> until
>>>>>>>>> > database
>>>>>>>>> > >> > migrations are completed.
>>>>>>>>> > >> >
>>>>>>>>> > >> > I have also migrated the "minimal" services viz
>>>>>>>>> provisioner[3],
>>>>>>>>> > >> rhythm[4],
>>>>>>>>> > >> > office[5], template[6] and customer[7] from MariaDB to
>>>>>>>>> PostgreSQL. The
>>>>>>>>> > >> > Spring datasource credentials had to be incorporated into
>>>>>>>>> > >> application.yaml
>>>>>>>>> > >> > files to make things work too.
>>>>>>>>> > >> >
>>>>>>>>> > >> > Given I updated service-starter [8] and demo-server [9],
>>>>>>>>> developers
>>>>>>>>> > can
>>>>>>>>> > >> > test the migrated services when running the
>>>>>>>>> *-Ddemoserver.lite=true*
>>>>>>>>> > >> option
>>>>>>>>> > >> > which builds the minimal set of services.
>>>>>>>>> > >> >
>>>>>>>>> > >> > You'll observe that some of the pull requests are heading
>>>>>>>>> to develop
>>>>>>>>> > >> > branches ( It appears comparing changes and creating PRs
>>>>>>>>> towards the
>>>>>>>>> > >> same
>>>>>>>>> > >> > branch name from origin to upstream expires after a few
>>>>>>>>> minutes on
>>>>>>>>> > >> > Github... weird )
>>>>>>>>> > >> >
>>>>>>>>> > >> > Here is a script [10] to enable anyone clone and do an
>>>>>>>>> > integration-test
>>>>>>>>> > >> of
>>>>>>>>> > >> > the new changes using demo-server.
>>>>>>>>> > >> >
>>>>>>>>> > >> > Kindly test these changes and provide feedback.
>>>>>>>>> > >> >
>>>>>>>>> > >> > At Your Service,
>>>>>>>>> > >> > Isaac Kamga.
>>>>>>>>> > >> >
>>>>>>>>> > >> > [1] https://github.com/apache/fineract-cn-postgresql/pull/1
>>>>>>>>> > >> > [2] https://github.com/apache/fineract-cn-test/pull/7
>>>>>>>>> > >> > [3]
>>>>>>>>> https://github.com/apache/fineract-cn-provisioner/pull/6
>>>>>>>>> > >> > [4] https://github.com/apache/fineract-cn-rhythm/pull/8
>>>>>>>>> > >> > [5] https://github.com/apache/fineract-cn-office/pull/9
>>>>>>>>> > >> > [6] https://github.com/apache/fineract-cn-template/pull/6
>>>>>>>>> > >> > [7] https://github.com/apache/fineract-cn-customer/pull/10
>>>>>>>>> > >> > [8]
>>>>>>>>> https://github.com/apache/fineract-cn-service-starter/pull/8
>>>>>>>>> > >> > [9]
>>>>>>>>> https://github.com/apache/fineract-cn-demo-server/pull/27
>>>>>>>>> > >> > [10]
>>>>>>>>> > >> >
>>>>>>>>> > >> >
>>>>>>>>> > >>
>>>>>>>>> >
>>>>>>>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>>>>>>>>> > >> >
>>>>>>>>> > >>
>>>>>>>>> > >
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>
Re: updates On Migration from MariaDB to PostgreSQL
Posted by Ebenezer Graham <eg...@alustudent.com>.
Hello Devs/Isaac,
I tested the service in both lite and full mode
Full mode resulted in an error
Complete log here: https://pastebin.com/J0YXvkLq
Time: 968.131
There was 1 failure:
1) startDevServer(org.apache.fineract.cn.dev.ServiceRunner)
java.lang.IllegalArgumentException: Index for header 'show' is 4 but
CSVRecord only has 4 values!
at org.apache.commons.csv.CSVRecord.get(CSVRecord.java:110)
at
org.apache.fineract.cn.accounting.importer.LedgerImporter.toLedger(LedgerImporter.java:106)
I suspect the error is caused during the parsing of i
mportdata/ledger-missing-name-case.csv as it has one missing value in row
1.
However, this is weird because, in the competent test, the test for i
mportdata/ledger-missing-name-case.csv doesn't fail but the test for
importdata/account-from-a-real-case.csv fails.
On top of that, the parser in LedgerImporter::importCSV uses the format
CSVFormat.RFC4180.withHeader() and so the import should work just fine.
I have been debugging it for a while now but no success in resolving the
issue. Let me know if you have had luck on your end.
Secondly, I did a black box testing in lite mode and got an SQLException
whenever I tried creating a customer or ledger and automatically logs out.
Looking forward to your response.
*At your service,*
*Ebenezer Graham*
*BSc (Hons) Computing*
[image: EmailSignature.png]
African Leadership University,
Power Mill Road, Pamplemousses,
Mauritius.
GitHub <https://github.com/ebenezergraham> | LinkedIn
<https://www.linkedin.com/in/ebenezer-graham/> | ebenezergraham.me
*Skype*:
ebenezer.graham <http://ebenezergraham.me/>
*This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the sender and then
delete this email. If you are not the intended recipient you are notified
that disclosing, copying, distributing or taking any action in reliance on
the contents of this information is strictly prohibited.*
On Fri, 10 May 2019 at 23:09, Isaac Kamga <is...@mifos.org> wrote:
> Hello Graham,
>
> Thanks for the feedback.
>
> Given the test was successful in lite mode, did you also check that the
> tables in databases seshat and playground in PostgreSQL were rightly
> created ?
>
> As for the error you encountered, I think it's not from the PostgreSQL
> setup since you've encountered it before in the MariaDB one.
>
> Kindly test with all services once more and share the entire error log
> using pastebin.
>
> At Your Service,
> Isaac Kamga.
>
> On Fri, May 10, 2019 at 4:06 AM Ebenezer Graham <eg...@alustudent.com>
> wrote:
>
>> Hi Isaac,
>>
>> Thanks for this effort.
>>
>> I have tried testing with all the services. but, I get the error below. *complete
>> log here: *https://pastebin.com/013aTJ8E
>>
>>
>> 1. 06:21:47.050 [Thread-13] INFO o.e.j.server.handler.ContextHandler
>> - Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@3e1fd62b
>> {/identity/v1,file:///tmp/jetty-docbase.1273617126957352135.2021/,UNAVAILABLE}
>> 2. E06:21:47.790 [DefaultMessageListenerContainer-1] ERROR
>> o.s.j.l.DefaultMessageListenerContainer - Could not refresh JMS Connection
>> for destination 'portfolio-v1' - retrying using FixedBackOff{interval=5000,
>> currentAttempts=0, maxAttempts=unlimited}. Cause: Error while attempting to
>> retrieve a connection from the pool; nested exception is
>> javax.jms.JMSException: Could not connect to broker URL:
>> tcp://localhost:61616. Reason: java.net.ConnectException: Connection
>> refused (Connection refused)
>>
>>
>>
>> I usually get this same error even with the MariaDB setup. I know its
>> related to the Message broker but I don't know how to resolve it or how to
>> avoid it. Can anyone share their experience on this issue?
>>
>> Also, I dropped the playground and seshat keyspaces in Cassandra as well
>> as in Postgres.
>> I changed the max-connection to 120 in PostgresSQL and also restarted the
>> system.
>>
>> I am convinced the problem is not with PostgreSQL or Cassandra, plus, it
>> worked a few hours ago when I started it in lite mode.
>>
>> PS: In light of this, how soon will this be merged into the develop
>> branches. I would love to completely switch to PostgreSQL. Toggling
>> between MariaDB and PostgreSQL is not easy :)
>>
>> *At your service,*
>> *Graham.*
>>
>> On Thu, 9 May 2019 at 20:13, Isaac Kamga <is...@mifos.org> wrote:
>>
>>> Hello Dev,
>>>
>>> Thanks very much for testing Graham
>>>
>>> I've raised pull requests for the Migration to PostgreSQL. Demo-server
>>> runs to completion successfully in both cases and databases and tables are
>>> rightly being created.
>>> I had to make some adjustments to default-setup's ledgers.csv file
>>> because of an error that came from the CSVParser.
>>>
>>> *Testing demo-server lite for minimal set of services*
>>> I have fixed the issue with connecting to the right databases. You can
>>> use the postgres-setup-lite.sh
>>> <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh>
>>> script to test the minimal set of libraries and services. After the run,
>>> you should be able to ascertain that databases are created and tables are
>>> created where they have to be.
>>>
>>> +Juhan Aasaru <aa...@gmail.com> please provide feedback how this test
>>> goes.
>>>
>>> *Testing demo-server for all services*
>>> Given that the connection issues have been resolved, it's time to do
>>> more testing;
>>> @Juhan, @Graham and anyone else, in order to test the migration in it's
>>> entirety using postgres-setup.sh script
>>> <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup.sh>,
>>> ensure to;
>>>
>>> 1. Install PostgreSQL
>>> 2. Change password for user postgres to postgres
>>> 3. Change max_connections to 120 in your postgresql.conf file ( usually
>>> in /etc/postgresql/... ) and restart the Postgresql service.Many
>>> connections to databases happening since all services are started.
>>> 4. Ensure Cassandra has no keyspaces seshat and playground
>>> 5. cd into a completely new directory from your usual Fineract CN
>>> development pwd
>>> 5. In integration-tests/fineract-cn-demo-server/build/libs/ and Run
>>> demo-server jar using *java -Ddemoserver.provision=true
>>> -Ddemoserver.persistent=true
>>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>> <http://127.0.0.1:9042/> -Dcassandra.cluster.user=cassandra
>>> -Dcassandra.cluster.pwd=password -Dcustom.postgresql.host=localhost
>>> -Dcustom.postgresql.user=postgres -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar*
>>>
>>> Kindly provide feedback on how the run goes in the both cases.
>>>
>>> At Your Service,
>>> Isaac Kamga.
>>>
>>> On Fri, May 3, 2019 at 7:21 PM Ebenezer Graham <eg...@alustudent.com>
>>> wrote:
>>>
>>>> Hello Isaac / Juhan
>>>>
>>>> I missed a number of relevant steps and this led to the initial failure.
>>>>
>>>> The test has been successful and I will list the conditions which led
>>>> to the successful run below.
>>>>
>>>> 1. I created a new directory and cloned the repositories using the
>>>> script you provided (
>>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>>>> )
>>>>
>>>> 2. I installed PostgreSQL server and changed the postgres user's
>>>> password to "postgres"
>>>>
>>>> 3. I used the command below to start the demo-server.
>>>>
>>>> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>>>> -Ddemoserver.lite=true -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>>> -Dcustom.postgresql.host=localhost -Dcustom.postgresql.user=postgres -jar
>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>
>>>> @Juhan, please try this approach and let's know the outcome.
>>>>
>>>> You should get an output similar to this.
>>>>
>>>> 18:11:28.717 [Thread-16] INFO o.e.jetty.server.AbstractConnector -
>>>> Stopped ServerConnector@59a94d0f{HTTP/1.1,[http/1.1]}{0.0.0.0:2020}
>>>> 18:11:28.720 [Thread-16] INFO o.e.j.server.handler.ContextHandler -
>>>> Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@1213ffbc
>>>> {/provisioner/v1,file:///tmp/jetty-docbase.8613597725021372791.2020/,UNAVAILABLE}
>>>> identity-v1 address:http://localhost:2021/identity/v1
>>>> office-v1 address:http://localhost:2023/office/v1
>>>> customer-v1 address:http://localhost:2024/customer/v1
>>>>
>>>> At your service
>>>>
>>>>
>>>> On Fri, 3 May 2019 at 21:09, Ebenezer Graham <eg...@alustudent.com>
>>>> wrote:
>>>>
>>>>> Hello Isaac/Juhan,
>>>>>
>>>>> I have tried testing the code in lite mode for 3 times and I got the error
>>>>> below.
>>>>> https://pastebin.com/TaJqSNjQ
>>>>>
>>>>> Before the failure, I got this info.
>>>>>
>>>>> 16:56:03.519 [qtp1262607708-22] INFO
>>>>> o.a.f.c.l.c.ServiceExceptionFilter - Responding with a service error
>>>>> ServiceError{code=409, message='The given identity instance didnt recognize
>>>>> the system token as valid. Perhaps the system keys for the provisioner or
>>>>> for the identity manager are misconfigured?'}
>>>>> 16:57:03.620 [qtp1262607708-20] INFO
>>>>> o.a.f.c.l.c.ServiceExceptionFilter - Responding with a service error
>>>>> ServiceError{code=409, message='The given identity instance didnt recognize
>>>>> the system token as valid. Perhaps the system keys for the provisioner or
>>>>> for the identity manager are misconfigured?'}
>>>>>
>>>>> I will keep testing to get more insights into the error.
>>>>>
>>>>> *At your service,*
>>>>>
>>>>> *Ebenezer Graham*
>>>>>
>>>>> *BSc (Hons) Computing*
>>>>>
>>>>>
>>>>> [image: EmailSignature.png]
>>>>>
>>>>> African Leadership University,
>>>>>
>>>>> Power Mill Road, Pamplemousses,
>>>>>
>>>>> Mauritius.
>>>>> GitHub <https://github.com/ebenezergraham> | LinkedIn
>>>>> <https://www.linkedin.com/in/ebenezer-graham/> | ebenezergraham.me
>>>>> *Skype*:
>>>>> ebenezer.graham <http://ebenezergraham.me/>
>>>>>
>>>>> *This email and any files transmitted with it are confidential and
>>>>> intended solely for the use of the individual or entity to whom they are
>>>>> addressed. If you have received this email in error please notify the
>>>>> sender and then delete this email. If you are not the intended recipient
>>>>> you are notified that disclosing, copying, distributing or taking any
>>>>> action in reliance on the contents of this information is strictly
>>>>> prohibited.*
>>>>>
>>>>>
>>>>> On Thu, 2 May 2019 at 06:38, Isaac Kamga <is...@mifos.org>
>>>>> wrote:
>>>>>
>>>>>> Hello Juhan,
>>>>>>
>>>>>> I've updated the PRs with commits to fix the errors you were
>>>>>> experiencing.
>>>>>>
>>>>>> Could you kindly wipe Cassandra and PostgreSQL clean, test once more
>>>>>> and provide feedback on how it goes ?
>>>>>>
>>>>>> Cheers,
>>>>>> Isaac Kamga.
>>>>>>
>>>>>> On Tue, Apr 9, 2019 at 4:36 PM Isaac Kamga <is...@mifos.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hey Juhan,
>>>>>>>
>>>>>>> Thanks for testing.
>>>>>>>
>>>>>>> Wipe clean your Cassandra keyspaces first.
>>>>>>>
>>>>>>> I suspect that what's interferring is the keyspaces from your last
>>>>>>> run when using MariaDB.
>>>>>>>
>>>>>>> Please provide feedback how it goes.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Isaac Kamga.
>>>>>>>
>>>>>>> On Sun, Mar 31, 2019 at 10:00 AM Juhan Aasaru <aa...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I Isaac,
>>>>>>>>
>>>>>>>> I pulled the changes and rebuilt the projects.
>>>>>>>>
>>>>>>>> Now when I run (postgres and cassandra working locally):
>>>>>>>>
>>>>>>>> java -Ddemoserver.persistent=true -Ddemoserver.provision=true
>>>>>>>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>>>>>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>>>>>>> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>>>>
>>>>>>>> I get an error here:
>>>>>>>>
>>>>>>>> 11:18:42.066 [main] INFO o.s.b.c.e.j.JettyEmbeddedServletContainer
>>>>>>>> - Jetty
>>>>>>>> started on port(s) 2024 (http/1.1)
>>>>>>>> 11:18:42.067 [main] INFO
>>>>>>>> o.s.c.n.e.EurekaDiscoveryClientConfiguration -
>>>>>>>> Updating port to 2024
>>>>>>>> 11:18:42.072 [main] INFO o.a.f.c.customer.CustomerApplication -
>>>>>>>> Started
>>>>>>>> CustomerApplication in 16.423 seconds (JVM running for 17.342)
>>>>>>>> 11:18:57.435 [qtp1479696465-21] INFO
>>>>>>>> o.a.f.c.l.c.ServiceExceptionFilter -
>>>>>>>> Responding with a service error ServiceError{code=409,
>>>>>>>> message='Application
>>>>>>>> identity-v1 already exists!'}
>>>>>>>> 11:18:57.464 [Thread-16] INFO
>>>>>>>> o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing
>>>>>>>>
>>>>>>>> org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@4d50efb8
>>>>>>>> :
>>>>>>>> startup date [Sun Mar 31 11:12:52 EEST 2019]; parent:
>>>>>>>>
>>>>>>>> org.springframework.context.annotation.AnnotationConfigApplicationContext@75a1cd57
>>>>>>>>
>>>>>>>> I understand I have to use -Ddemoserver.provision=true flag to
>>>>>>>> provision
>>>>>>>> the initial database so I won't be able to take that one out.
>>>>>>>>
>>>>>>>> Juhan
>>>>>>>>
>>>>>>>>
>>>>>>>> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval
>>>>>>>> R, 29.
>>>>>>>> märts 2019 kell 04:17:
>>>>>>>>
>>>>>>>> > Hey Juhan,
>>>>>>>> >
>>>>>>>> > I have updated the Pull request to fineract-cn-provisioner with a
>>>>>>>> commit
>>>>>>>> > [1] which fixes the errors you're experiencing.
>>>>>>>> >
>>>>>>>> > I'll advise that you test using option
>>>>>>>> -Ddemoserver.persistent=true and
>>>>>>>> > -Ddemoserver.lite=true and we're testing just the minimal set of
>>>>>>>> services
>>>>>>>> > now. In fact,I tested and the tail of my logs showed thus
>>>>>>>> > *identity-v1 address:http://localhost:2021/identity/v1
>>>>>>>> > <
>>>>>>>> >
>>>>>>>> https://www.google.com/url?q=http://localhost:2021/identity/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNEjCID96mw3LF1ZIPmjEIQkHh_LcQ
>>>>>>>> > >*
>>>>>>>> >
>>>>>>>> > *office-v1 address:http://localhost:2023/office/v1
>>>>>>>> > <
>>>>>>>> >
>>>>>>>> https://www.google.com/url?q=http://localhost:2023/office/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNGjyk1BHroLddOTd0FhJUJG4zdVag
>>>>>>>> > >customer-v1
>>>>>>>> > address:http://localhost:2024/customer/v1
>>>>>>>> > <
>>>>>>>> >
>>>>>>>> https://www.google.com/url?q=http://localhost:2024/customer/v1&sa=D&source=hangouts&ust=1553910362599000&usg=AFQjCNGfcoJZIxmcsX34YeEXRUzS0oC2hA
>>>>>>>> > >*
>>>>>>>> > Albeit, this error [2] turned up related to the Notifications
>>>>>>>> service. Just
>>>>>>>> > ignore it for now as it will be handled down the line during
>>>>>>>> extensive
>>>>>>>> > integration testing.
>>>>>>>> >
>>>>>>>> > We can keep testing using -Ddemoserver.provision=true for later
>>>>>>>> too when
>>>>>>>> > we've migrated and tested the individual services.
>>>>>>>> >
>>>>>>>> > Can you kindly update your provisioner repository and test once
>>>>>>>> more and
>>>>>>>> > let's keep fixing ?
>>>>>>>> >
>>>>>>>> > [1]
>>>>>>>> >
>>>>>>>> >
>>>>>>>> https://github.com/Izakey/fineract-cn-provisioner/commit/b83331b723f175003ccf069764414e4ab371662c
>>>>>>>> >
>>>>>>>> > [2] https://pastebin.com/DBzNyHir
>>>>>>>> >
>>>>>>>> > Cheers,
>>>>>>>> > Isaac Kamga.
>>>>>>>> >
>>>>>>>> > On Wed, Mar 27, 2019 at 10:23 PM Isaac Kamga <
>>>>>>>> isaac.kamga@mifos.org>
>>>>>>>> > wrote:
>>>>>>>> >
>>>>>>>> > > Hello Juhan,
>>>>>>>> > >
>>>>>>>> > > Thank you for taking the time to do some testing.
>>>>>>>> > >
>>>>>>>> > > On Wed, Mar 27, 2019 at 10:56 AM Juhan Aasaru <aa...@gmail.com>
>>>>>>>> wrote:
>>>>>>>> > >
>>>>>>>> > >> Hi Isaac,
>>>>>>>> > >>
>>>>>>>> > >> great initiative towards pushing to PostgreSQL.
>>>>>>>> > >>
>>>>>>>> > >> I looked at your instructions and tried to run your changes
>>>>>>>> locally.
>>>>>>>> > >> The sh script came in handy to quickly clone and build
>>>>>>>> everything that
>>>>>>>> > was
>>>>>>>> > >> needed.
>>>>>>>> > >> But then I got confused about provisioning.
>>>>>>>> > >>
>>>>>>>> > >> Question #1: How exactly I should run demo-server? I tried the
>>>>>>>> following
>>>>>>>> > >> options:
>>>>>>>> > >>
>>>>>>>> > >> 1. I didn't have postgre and cassandra running locally.
>>>>>>>> > >> java -jar -Ddemoserver.lite=true
>>>>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>>>> > >>
>>>>>>>> > >> resulted
>>>>>>>> > >>
>>>>>>>> > >> org.springframework.beans.factory.BeanCreationException: Error
>>>>>>>> creating
>>>>>>>> > >> bean with name 'provisionerInitializer':
>>>>>>>> > >> Invocation of init method failed; nested exception is
>>>>>>>> > >> java.lang.IllegalStateException: Could not initialize service!
>>>>>>>> > >> Caused by: org.postgresql.util.PSQLException: FATAL: database
>>>>>>>> "seshat"
>>>>>>>> > >> does
>>>>>>>> > >> not exist
>>>>>>>> > >>
>>>>>>>> > >
>>>>>>>> > > I haven't done testing using this option yet.
>>>>>>>> > >
>>>>>>>> > >
>>>>>>>> > >>
>>>>>>>> > >> 2. I started posgresql (and created a database seshat) and
>>>>>>>> cassandra and
>>>>>>>> > >> ran:
>>>>>>>> > >> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>>>>>>>> > >> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 \
>>>>>>>> > >> -Dcassandra.cluster.user=cassandra
>>>>>>>> -Dcassandra.cluster.pwd=password
>>>>>>>> > >> -Ddemoserver.lite=true -jar
>>>>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>>>> > >>
>>>>>>>> > >> Initially applications started up one after anther
>>>>>>>> > >> (ProvisionerApplication,
>>>>>>>> > >> RhythmApplication, OfficeApplication)
>>>>>>>> > >> but CustomerApplication threw an exception:
>>>>>>>> > >>
>>>>>>>> > >> 11:50:39.377 [qtp1424482154-21] WARN
>>>>>>>> o.e.jetty.servlet.ServletHandler -
>>>>>>>> > >> org.springframework.web.util.NestedServletException: Request
>>>>>>>> processing
>>>>>>>> > >> failed; nested exception is java.lang.IllegalStateException:
>>>>>>>> Could not
>>>>>>>> > >> insert org.apache.fineract.cn.provisioner.tenant info!
>>>>>>>> > >> Caused by: org.postgresql.util.PSQLException: ERROR: relation
>>>>>>>> > >> "seshat.tenants" does not exist
>>>>>>>> > >>
>>>>>>>> > >
>>>>>>>> > > Try running the same command above without the provision=true
>>>>>>>> option and
>>>>>>>> > > share what you get.
>>>>>>>> > >
>>>>>>>> > > I had to create database seshat in postgresql and the can't
>>>>>>>> find database
>>>>>>>> > > error went away. Down the road, also encountered the same error
>>>>>>>> > > <https://pastebin.com/qCT1T5KM> "Could not insert ....
>>>>>>>> > provisioner.tenant
>>>>>>>> > > info !"
>>>>>>>> > > I'm currently working on resolving this.
>>>>>>>> > >
>>>>>>>> > >
>>>>>>>> > >>
>>>>>>>> > >> Question #2 - do I still need to have Mysql running? I see
>>>>>>>> that you
>>>>>>>> > >> haven't
>>>>>>>> > >> made code changes to identity
>>>>>>>> > >> which as I understand still uses Mysql.
>>>>>>>> > >>
>>>>>>>> > >
>>>>>>>> > > I don't think you need MySQL running to test the migrations
>>>>>>>> currently
>>>>>>>> > > underway and there was no need for code changes in the identity
>>>>>>>> service.
>>>>>>>> > > However, it could be helpful to see the structure of the
>>>>>>>> databases in
>>>>>>>> > MySQL
>>>>>>>> > > so that it helps with debugging and testing.
>>>>>>>> > >
>>>>>>>> > > Whenever you're available, tell me so we schedule a call so I
>>>>>>>> look into
>>>>>>>> > > what your experiencing too.
>>>>>>>> > >
>>>>>>>> > > Cheers,
>>>>>>>> > > Isaac Kamga.
>>>>>>>> > >
>>>>>>>> > >
>>>>>>>> > >>
>>>>>>>> > >> Kind regards
>>>>>>>> > >> Juhan
>>>>>>>> > >>
>>>>>>>> > >>
>>>>>>>> > >> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas
>>>>>>>> kuupäeval L, 23.
>>>>>>>> > >> märts 2019 kell 04:31:
>>>>>>>> > >>
>>>>>>>> > >> > Hello fineracters,
>>>>>>>> > >> >
>>>>>>>> > >> > Trust that you had a great week.
>>>>>>>> > >> >
>>>>>>>> > >> > I just finished migrating libraries PostgreSQL[1] and
>>>>>>>> Test[2] from
>>>>>>>> > >> MariaDB
>>>>>>>> > >> > to PostgreSQL, mostly adapting SQL statements & database
>>>>>>>> connections
>>>>>>>> > to
>>>>>>>> > >> > work with PostgreSQL and reverting back to Hibernate ORM
>>>>>>>> until
>>>>>>>> > database
>>>>>>>> > >> > migrations are completed.
>>>>>>>> > >> >
>>>>>>>> > >> > I have also migrated the "minimal" services viz
>>>>>>>> provisioner[3],
>>>>>>>> > >> rhythm[4],
>>>>>>>> > >> > office[5], template[6] and customer[7] from MariaDB to
>>>>>>>> PostgreSQL. The
>>>>>>>> > >> > Spring datasource credentials had to be incorporated into
>>>>>>>> > >> application.yaml
>>>>>>>> > >> > files to make things work too.
>>>>>>>> > >> >
>>>>>>>> > >> > Given I updated service-starter [8] and demo-server [9],
>>>>>>>> developers
>>>>>>>> > can
>>>>>>>> > >> > test the migrated services when running the
>>>>>>>> *-Ddemoserver.lite=true*
>>>>>>>> > >> option
>>>>>>>> > >> > which builds the minimal set of services.
>>>>>>>> > >> >
>>>>>>>> > >> > You'll observe that some of the pull requests are heading to
>>>>>>>> develop
>>>>>>>> > >> > branches ( It appears comparing changes and creating PRs
>>>>>>>> towards the
>>>>>>>> > >> same
>>>>>>>> > >> > branch name from origin to upstream expires after a few
>>>>>>>> minutes on
>>>>>>>> > >> > Github... weird )
>>>>>>>> > >> >
>>>>>>>> > >> > Here is a script [10] to enable anyone clone and do an
>>>>>>>> > integration-test
>>>>>>>> > >> of
>>>>>>>> > >> > the new changes using demo-server.
>>>>>>>> > >> >
>>>>>>>> > >> > Kindly test these changes and provide feedback.
>>>>>>>> > >> >
>>>>>>>> > >> > At Your Service,
>>>>>>>> > >> > Isaac Kamga.
>>>>>>>> > >> >
>>>>>>>> > >> > [1] https://github.com/apache/fineract-cn-postgresql/pull/1
>>>>>>>> > >> > [2] https://github.com/apache/fineract-cn-test/pull/7
>>>>>>>> > >> > [3] https://github.com/apache/fineract-cn-provisioner/pull/6
>>>>>>>> > >> > [4] https://github.com/apache/fineract-cn-rhythm/pull/8
>>>>>>>> > >> > [5] https://github.com/apache/fineract-cn-office/pull/9
>>>>>>>> > >> > [6] https://github.com/apache/fineract-cn-template/pull/6
>>>>>>>> > >> > [7] https://github.com/apache/fineract-cn-customer/pull/10
>>>>>>>> > >> > [8]
>>>>>>>> https://github.com/apache/fineract-cn-service-starter/pull/8
>>>>>>>> > >> > [9]
>>>>>>>> https://github.com/apache/fineract-cn-demo-server/pull/27
>>>>>>>> > >> > [10]
>>>>>>>> > >> >
>>>>>>>> > >> >
>>>>>>>> > >>
>>>>>>>> >
>>>>>>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>>>>>>>> > >> >
>>>>>>>> > >>
>>>>>>>> > >
>>>>>>>> >
>>>>>>>>
>>>>>>>
Re: updates On Migration from MariaDB to PostgreSQL
Posted by Isaac Kamga <is...@mifos.org>.
Hello Graham,
Thanks for the feedback.
Given the test was successful in lite mode, did you also check that the
tables in databases seshat and playground in PostgreSQL were rightly
created ?
As for the error you encountered, I think it's not from the PostgreSQL
setup since you've encountered it before in the MariaDB one.
Kindly test with all services once more and share the entire error log
using pastebin.
At Your Service,
Isaac Kamga.
On Fri, May 10, 2019 at 4:06 AM Ebenezer Graham <eg...@alustudent.com>
wrote:
> Hi Isaac,
>
> Thanks for this effort.
>
> I have tried testing with all the services. but, I get the error below. *complete
> log here: *https://pastebin.com/013aTJ8E
>
>
> 1. 06:21:47.050 [Thread-13] INFO o.e.j.server.handler.ContextHandler
> - Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@3e1fd62b
> {/identity/v1,file:///tmp/jetty-docbase.1273617126957352135.2021/,UNAVAILABLE}
> 2. E06:21:47.790 [DefaultMessageListenerContainer-1] ERROR
> o.s.j.l.DefaultMessageListenerContainer - Could not refresh JMS Connection
> for destination 'portfolio-v1' - retrying using FixedBackOff{interval=5000,
> currentAttempts=0, maxAttempts=unlimited}. Cause: Error while attempting to
> retrieve a connection from the pool; nested exception is
> javax.jms.JMSException: Could not connect to broker URL:
> tcp://localhost:61616. Reason: java.net.ConnectException: Connection
> refused (Connection refused)
>
>
>
> I usually get this same error even with the MariaDB setup. I know its
> related to the Message broker but I don't know how to resolve it or how to
> avoid it. Can anyone share their experience on this issue?
>
> Also, I dropped the playground and seshat keyspaces in Cassandra as well
> as in Postgres.
> I changed the max-connection to 120 in PostgresSQL and also restarted the
> system.
>
> I am convinced the problem is not with PostgreSQL or Cassandra, plus, it
> worked a few hours ago when I started it in lite mode.
>
> PS: In light of this, how soon will this be merged into the develop
> branches. I would love to completely switch to PostgreSQL. Toggling
> between MariaDB and PostgreSQL is not easy :)
>
> *At your service,*
> *Graham.*
>
> On Thu, 9 May 2019 at 20:13, Isaac Kamga <is...@mifos.org> wrote:
>
>> Hello Dev,
>>
>> Thanks very much for testing Graham
>>
>> I've raised pull requests for the Migration to PostgreSQL. Demo-server
>> runs to completion successfully in both cases and databases and tables are
>> rightly being created.
>> I had to make some adjustments to default-setup's ledgers.csv file
>> because of an error that came from the CSVParser.
>>
>> *Testing demo-server lite for minimal set of services*
>> I have fixed the issue with connecting to the right databases. You can
>> use the postgres-setup-lite.sh
>> <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh>
>> script to test the minimal set of libraries and services. After the run,
>> you should be able to ascertain that databases are created and tables are
>> created where they have to be.
>>
>> +Juhan Aasaru <aa...@gmail.com> please provide feedback how this test
>> goes.
>>
>> *Testing demo-server for all services*
>> Given that the connection issues have been resolved, it's time to do more
>> testing;
>> @Juhan, @Graham and anyone else, in order to test the migration in it's
>> entirety using postgres-setup.sh script
>> <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup.sh>,
>> ensure to;
>>
>> 1. Install PostgreSQL
>> 2. Change password for user postgres to postgres
>> 3. Change max_connections to 120 in your postgresql.conf file ( usually
>> in /etc/postgresql/... ) and restart the Postgresql service.Many
>> connections to databases happening since all services are started.
>> 4. Ensure Cassandra has no keyspaces seshat and playground
>> 5. cd into a completely new directory from your usual Fineract CN
>> development pwd
>> 5. In integration-tests/fineract-cn-demo-server/build/libs/ and Run
>> demo-server jar using *java -Ddemoserver.provision=true
>> -Ddemoserver.persistent=true
>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>> <http://127.0.0.1:9042/> -Dcassandra.cluster.user=cassandra
>> -Dcassandra.cluster.pwd=password -Dcustom.postgresql.host=localhost
>> -Dcustom.postgresql.user=postgres -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar*
>>
>> Kindly provide feedback on how the run goes in the both cases.
>>
>> At Your Service,
>> Isaac Kamga.
>>
>> On Fri, May 3, 2019 at 7:21 PM Ebenezer Graham <eg...@alustudent.com>
>> wrote:
>>
>>> Hello Isaac / Juhan
>>>
>>> I missed a number of relevant steps and this led to the initial failure.
>>>
>>> The test has been successful and I will list the conditions which led to
>>> the successful run below.
>>>
>>> 1. I created a new directory and cloned the repositories using the
>>> script you provided (
>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>>> )
>>>
>>> 2. I installed PostgreSQL server and changed the postgres user's
>>> password to "postgres"
>>>
>>> 3. I used the command below to start the demo-server.
>>>
>>> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>>> -Ddemoserver.lite=true -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>> -Dcustom.postgresql.host=localhost -Dcustom.postgresql.user=postgres -jar
>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>
>>> @Juhan, please try this approach and let's know the outcome.
>>>
>>> You should get an output similar to this.
>>>
>>> 18:11:28.717 [Thread-16] INFO o.e.jetty.server.AbstractConnector -
>>> Stopped ServerConnector@59a94d0f{HTTP/1.1,[http/1.1]}{0.0.0.0:2020}
>>> 18:11:28.720 [Thread-16] INFO o.e.j.server.handler.ContextHandler -
>>> Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@1213ffbc
>>> {/provisioner/v1,file:///tmp/jetty-docbase.8613597725021372791.2020/,UNAVAILABLE}
>>> identity-v1 address:http://localhost:2021/identity/v1
>>> office-v1 address:http://localhost:2023/office/v1
>>> customer-v1 address:http://localhost:2024/customer/v1
>>>
>>> At your service
>>>
>>>
>>> On Fri, 3 May 2019 at 21:09, Ebenezer Graham <eg...@alustudent.com>
>>> wrote:
>>>
>>>> Hello Isaac/Juhan,
>>>>
>>>> I have tried testing the code in lite mode for 3 times and I got the error
>>>> below.
>>>> https://pastebin.com/TaJqSNjQ
>>>>
>>>> Before the failure, I got this info.
>>>>
>>>> 16:56:03.519 [qtp1262607708-22] INFO
>>>> o.a.f.c.l.c.ServiceExceptionFilter - Responding with a service error
>>>> ServiceError{code=409, message='The given identity instance didnt recognize
>>>> the system token as valid. Perhaps the system keys for the provisioner or
>>>> for the identity manager are misconfigured?'}
>>>> 16:57:03.620 [qtp1262607708-20] INFO
>>>> o.a.f.c.l.c.ServiceExceptionFilter - Responding with a service error
>>>> ServiceError{code=409, message='The given identity instance didnt recognize
>>>> the system token as valid. Perhaps the system keys for the provisioner or
>>>> for the identity manager are misconfigured?'}
>>>>
>>>> I will keep testing to get more insights into the error.
>>>>
>>>> *At your service,*
>>>>
>>>> *Ebenezer Graham*
>>>>
>>>> *BSc (Hons) Computing*
>>>>
>>>>
>>>> [image: EmailSignature.png]
>>>>
>>>> African Leadership University,
>>>>
>>>> Power Mill Road, Pamplemousses,
>>>>
>>>> Mauritius.
>>>> GitHub <https://github.com/ebenezergraham> | LinkedIn
>>>> <https://www.linkedin.com/in/ebenezer-graham/> | ebenezergraham.me
>>>> *Skype*:
>>>> ebenezer.graham <http://ebenezergraham.me/>
>>>>
>>>> *This email and any files transmitted with it are confidential and
>>>> intended solely for the use of the individual or entity to whom they are
>>>> addressed. If you have received this email in error please notify the
>>>> sender and then delete this email. If you are not the intended recipient
>>>> you are notified that disclosing, copying, distributing or taking any
>>>> action in reliance on the contents of this information is strictly
>>>> prohibited.*
>>>>
>>>>
>>>> On Thu, 2 May 2019 at 06:38, Isaac Kamga <is...@mifos.org> wrote:
>>>>
>>>>> Hello Juhan,
>>>>>
>>>>> I've updated the PRs with commits to fix the errors you were
>>>>> experiencing.
>>>>>
>>>>> Could you kindly wipe Cassandra and PostgreSQL clean, test once more
>>>>> and provide feedback on how it goes ?
>>>>>
>>>>> Cheers,
>>>>> Isaac Kamga.
>>>>>
>>>>> On Tue, Apr 9, 2019 at 4:36 PM Isaac Kamga <is...@mifos.org>
>>>>> wrote:
>>>>>
>>>>>> Hey Juhan,
>>>>>>
>>>>>> Thanks for testing.
>>>>>>
>>>>>> Wipe clean your Cassandra keyspaces first.
>>>>>>
>>>>>> I suspect that what's interferring is the keyspaces from your last
>>>>>> run when using MariaDB.
>>>>>>
>>>>>> Please provide feedback how it goes.
>>>>>>
>>>>>> Cheers,
>>>>>> Isaac Kamga.
>>>>>>
>>>>>> On Sun, Mar 31, 2019 at 10:00 AM Juhan Aasaru <aa...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I Isaac,
>>>>>>>
>>>>>>> I pulled the changes and rebuilt the projects.
>>>>>>>
>>>>>>> Now when I run (postgres and cassandra working locally):
>>>>>>>
>>>>>>> java -Ddemoserver.persistent=true -Ddemoserver.provision=true
>>>>>>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>>>>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>>>>>> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>>>
>>>>>>> I get an error here:
>>>>>>>
>>>>>>> 11:18:42.066 [main] INFO o.s.b.c.e.j.JettyEmbeddedServletContainer
>>>>>>> - Jetty
>>>>>>> started on port(s) 2024 (http/1.1)
>>>>>>> 11:18:42.067 [main] INFO
>>>>>>> o.s.c.n.e.EurekaDiscoveryClientConfiguration -
>>>>>>> Updating port to 2024
>>>>>>> 11:18:42.072 [main] INFO o.a.f.c.customer.CustomerApplication -
>>>>>>> Started
>>>>>>> CustomerApplication in 16.423 seconds (JVM running for 17.342)
>>>>>>> 11:18:57.435 [qtp1479696465-21] INFO
>>>>>>> o.a.f.c.l.c.ServiceExceptionFilter -
>>>>>>> Responding with a service error ServiceError{code=409,
>>>>>>> message='Application
>>>>>>> identity-v1 already exists!'}
>>>>>>> 11:18:57.464 [Thread-16] INFO
>>>>>>> o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing
>>>>>>>
>>>>>>> org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@4d50efb8
>>>>>>> :
>>>>>>> startup date [Sun Mar 31 11:12:52 EEST 2019]; parent:
>>>>>>>
>>>>>>> org.springframework.context.annotation.AnnotationConfigApplicationContext@75a1cd57
>>>>>>>
>>>>>>> I understand I have to use -Ddemoserver.provision=true flag to
>>>>>>> provision
>>>>>>> the initial database so I won't be able to take that one out.
>>>>>>>
>>>>>>> Juhan
>>>>>>>
>>>>>>>
>>>>>>> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval R,
>>>>>>> 29.
>>>>>>> märts 2019 kell 04:17:
>>>>>>>
>>>>>>> > Hey Juhan,
>>>>>>> >
>>>>>>> > I have updated the Pull request to fineract-cn-provisioner with a
>>>>>>> commit
>>>>>>> > [1] which fixes the errors you're experiencing.
>>>>>>> >
>>>>>>> > I'll advise that you test using option
>>>>>>> -Ddemoserver.persistent=true and
>>>>>>> > -Ddemoserver.lite=true and we're testing just the minimal set of
>>>>>>> services
>>>>>>> > now. In fact,I tested and the tail of my logs showed thus
>>>>>>> > *identity-v1 address:http://localhost:2021/identity/v1
>>>>>>> > <
>>>>>>> >
>>>>>>> https://www.google.com/url?q=http://localhost:2021/identity/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNEjCID96mw3LF1ZIPmjEIQkHh_LcQ
>>>>>>> > >*
>>>>>>> >
>>>>>>> > *office-v1 address:http://localhost:2023/office/v1
>>>>>>> > <
>>>>>>> >
>>>>>>> https://www.google.com/url?q=http://localhost:2023/office/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNGjyk1BHroLddOTd0FhJUJG4zdVag
>>>>>>> > >customer-v1
>>>>>>> > address:http://localhost:2024/customer/v1
>>>>>>> > <
>>>>>>> >
>>>>>>> https://www.google.com/url?q=http://localhost:2024/customer/v1&sa=D&source=hangouts&ust=1553910362599000&usg=AFQjCNGfcoJZIxmcsX34YeEXRUzS0oC2hA
>>>>>>> > >*
>>>>>>> > Albeit, this error [2] turned up related to the Notifications
>>>>>>> service. Just
>>>>>>> > ignore it for now as it will be handled down the line during
>>>>>>> extensive
>>>>>>> > integration testing.
>>>>>>> >
>>>>>>> > We can keep testing using -Ddemoserver.provision=true for later
>>>>>>> too when
>>>>>>> > we've migrated and tested the individual services.
>>>>>>> >
>>>>>>> > Can you kindly update your provisioner repository and test once
>>>>>>> more and
>>>>>>> > let's keep fixing ?
>>>>>>> >
>>>>>>> > [1]
>>>>>>> >
>>>>>>> >
>>>>>>> https://github.com/Izakey/fineract-cn-provisioner/commit/b83331b723f175003ccf069764414e4ab371662c
>>>>>>> >
>>>>>>> > [2] https://pastebin.com/DBzNyHir
>>>>>>> >
>>>>>>> > Cheers,
>>>>>>> > Isaac Kamga.
>>>>>>> >
>>>>>>> > On Wed, Mar 27, 2019 at 10:23 PM Isaac Kamga <
>>>>>>> isaac.kamga@mifos.org>
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> > > Hello Juhan,
>>>>>>> > >
>>>>>>> > > Thank you for taking the time to do some testing.
>>>>>>> > >
>>>>>>> > > On Wed, Mar 27, 2019 at 10:56 AM Juhan Aasaru <aa...@gmail.com>
>>>>>>> wrote:
>>>>>>> > >
>>>>>>> > >> Hi Isaac,
>>>>>>> > >>
>>>>>>> > >> great initiative towards pushing to PostgreSQL.
>>>>>>> > >>
>>>>>>> > >> I looked at your instructions and tried to run your changes
>>>>>>> locally.
>>>>>>> > >> The sh script came in handy to quickly clone and build
>>>>>>> everything that
>>>>>>> > was
>>>>>>> > >> needed.
>>>>>>> > >> But then I got confused about provisioning.
>>>>>>> > >>
>>>>>>> > >> Question #1: How exactly I should run demo-server? I tried the
>>>>>>> following
>>>>>>> > >> options:
>>>>>>> > >>
>>>>>>> > >> 1. I didn't have postgre and cassandra running locally.
>>>>>>> > >> java -jar -Ddemoserver.lite=true
>>>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>>> > >>
>>>>>>> > >> resulted
>>>>>>> > >>
>>>>>>> > >> org.springframework.beans.factory.BeanCreationException: Error
>>>>>>> creating
>>>>>>> > >> bean with name 'provisionerInitializer':
>>>>>>> > >> Invocation of init method failed; nested exception is
>>>>>>> > >> java.lang.IllegalStateException: Could not initialize service!
>>>>>>> > >> Caused by: org.postgresql.util.PSQLException: FATAL: database
>>>>>>> "seshat"
>>>>>>> > >> does
>>>>>>> > >> not exist
>>>>>>> > >>
>>>>>>> > >
>>>>>>> > > I haven't done testing using this option yet.
>>>>>>> > >
>>>>>>> > >
>>>>>>> > >>
>>>>>>> > >> 2. I started posgresql (and created a database seshat) and
>>>>>>> cassandra and
>>>>>>> > >> ran:
>>>>>>> > >> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>>>>>>> > >> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 \
>>>>>>> > >> -Dcassandra.cluster.user=cassandra
>>>>>>> -Dcassandra.cluster.pwd=password
>>>>>>> > >> -Ddemoserver.lite=true -jar
>>>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>>> > >>
>>>>>>> > >> Initially applications started up one after anther
>>>>>>> > >> (ProvisionerApplication,
>>>>>>> > >> RhythmApplication, OfficeApplication)
>>>>>>> > >> but CustomerApplication threw an exception:
>>>>>>> > >>
>>>>>>> > >> 11:50:39.377 [qtp1424482154-21] WARN
>>>>>>> o.e.jetty.servlet.ServletHandler -
>>>>>>> > >> org.springframework.web.util.NestedServletException: Request
>>>>>>> processing
>>>>>>> > >> failed; nested exception is java.lang.IllegalStateException:
>>>>>>> Could not
>>>>>>> > >> insert org.apache.fineract.cn.provisioner.tenant info!
>>>>>>> > >> Caused by: org.postgresql.util.PSQLException: ERROR: relation
>>>>>>> > >> "seshat.tenants" does not exist
>>>>>>> > >>
>>>>>>> > >
>>>>>>> > > Try running the same command above without the provision=true
>>>>>>> option and
>>>>>>> > > share what you get.
>>>>>>> > >
>>>>>>> > > I had to create database seshat in postgresql and the can't find
>>>>>>> database
>>>>>>> > > error went away. Down the road, also encountered the same error
>>>>>>> > > <https://pastebin.com/qCT1T5KM> "Could not insert ....
>>>>>>> > provisioner.tenant
>>>>>>> > > info !"
>>>>>>> > > I'm currently working on resolving this.
>>>>>>> > >
>>>>>>> > >
>>>>>>> > >>
>>>>>>> > >> Question #2 - do I still need to have Mysql running? I see that
>>>>>>> you
>>>>>>> > >> haven't
>>>>>>> > >> made code changes to identity
>>>>>>> > >> which as I understand still uses Mysql.
>>>>>>> > >>
>>>>>>> > >
>>>>>>> > > I don't think you need MySQL running to test the migrations
>>>>>>> currently
>>>>>>> > > underway and there was no need for code changes in the identity
>>>>>>> service.
>>>>>>> > > However, it could be helpful to see the structure of the
>>>>>>> databases in
>>>>>>> > MySQL
>>>>>>> > > so that it helps with debugging and testing.
>>>>>>> > >
>>>>>>> > > Whenever you're available, tell me so we schedule a call so I
>>>>>>> look into
>>>>>>> > > what your experiencing too.
>>>>>>> > >
>>>>>>> > > Cheers,
>>>>>>> > > Isaac Kamga.
>>>>>>> > >
>>>>>>> > >
>>>>>>> > >>
>>>>>>> > >> Kind regards
>>>>>>> > >> Juhan
>>>>>>> > >>
>>>>>>> > >>
>>>>>>> > >> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas
>>>>>>> kuupäeval L, 23.
>>>>>>> > >> märts 2019 kell 04:31:
>>>>>>> > >>
>>>>>>> > >> > Hello fineracters,
>>>>>>> > >> >
>>>>>>> > >> > Trust that you had a great week.
>>>>>>> > >> >
>>>>>>> > >> > I just finished migrating libraries PostgreSQL[1] and Test[2]
>>>>>>> from
>>>>>>> > >> MariaDB
>>>>>>> > >> > to PostgreSQL, mostly adapting SQL statements & database
>>>>>>> connections
>>>>>>> > to
>>>>>>> > >> > work with PostgreSQL and reverting back to Hibernate ORM until
>>>>>>> > database
>>>>>>> > >> > migrations are completed.
>>>>>>> > >> >
>>>>>>> > >> > I have also migrated the "minimal" services viz
>>>>>>> provisioner[3],
>>>>>>> > >> rhythm[4],
>>>>>>> > >> > office[5], template[6] and customer[7] from MariaDB to
>>>>>>> PostgreSQL. The
>>>>>>> > >> > Spring datasource credentials had to be incorporated into
>>>>>>> > >> application.yaml
>>>>>>> > >> > files to make things work too.
>>>>>>> > >> >
>>>>>>> > >> > Given I updated service-starter [8] and demo-server [9],
>>>>>>> developers
>>>>>>> > can
>>>>>>> > >> > test the migrated services when running the
>>>>>>> *-Ddemoserver.lite=true*
>>>>>>> > >> option
>>>>>>> > >> > which builds the minimal set of services.
>>>>>>> > >> >
>>>>>>> > >> > You'll observe that some of the pull requests are heading to
>>>>>>> develop
>>>>>>> > >> > branches ( It appears comparing changes and creating PRs
>>>>>>> towards the
>>>>>>> > >> same
>>>>>>> > >> > branch name from origin to upstream expires after a few
>>>>>>> minutes on
>>>>>>> > >> > Github... weird )
>>>>>>> > >> >
>>>>>>> > >> > Here is a script [10] to enable anyone clone and do an
>>>>>>> > integration-test
>>>>>>> > >> of
>>>>>>> > >> > the new changes using demo-server.
>>>>>>> > >> >
>>>>>>> > >> > Kindly test these changes and provide feedback.
>>>>>>> > >> >
>>>>>>> > >> > At Your Service,
>>>>>>> > >> > Isaac Kamga.
>>>>>>> > >> >
>>>>>>> > >> > [1] https://github.com/apache/fineract-cn-postgresql/pull/1
>>>>>>> > >> > [2] https://github.com/apache/fineract-cn-test/pull/7
>>>>>>> > >> > [3] https://github.com/apache/fineract-cn-provisioner/pull/6
>>>>>>> > >> > [4] https://github.com/apache/fineract-cn-rhythm/pull/8
>>>>>>> > >> > [5] https://github.com/apache/fineract-cn-office/pull/9
>>>>>>> > >> > [6] https://github.com/apache/fineract-cn-template/pull/6
>>>>>>> > >> > [7] https://github.com/apache/fineract-cn-customer/pull/10
>>>>>>> > >> > [8]
>>>>>>> https://github.com/apache/fineract-cn-service-starter/pull/8
>>>>>>> > >> > [9] https://github.com/apache/fineract-cn-demo-server/pull/27
>>>>>>> > >> > [10]
>>>>>>> > >> >
>>>>>>> > >> >
>>>>>>> > >>
>>>>>>> >
>>>>>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>>>>>>> > >> >
>>>>>>> > >>
>>>>>>> > >
>>>>>>> >
>>>>>>>
>>>>>>
Re: updates On Migration from MariaDB to PostgreSQL
Posted by Ebenezer Graham <eg...@alustudent.com>.
Hi Isaac,
Thanks for this effort.
I have tried testing with all the services. but, I get the error
below. *complete
log here: *https://pastebin.com/013aTJ8E
1. 06:21:47.050 [Thread-13] INFO o.e.j.server.handler.ContextHandler -
Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@3e1fd62b
{/identity/v1,file:///tmp/jetty-docbase.1273617126957352135.2021/,UNAVAILABLE}
2. E06:21:47.790 [DefaultMessageListenerContainer-1] ERROR
o.s.j.l.DefaultMessageListenerContainer - Could not refresh JMS Connection
for destination 'portfolio-v1' - retrying using FixedBackOff{interval=5000,
currentAttempts=0, maxAttempts=unlimited}. Cause: Error while attempting to
retrieve a connection from the pool; nested exception is
javax.jms.JMSException: Could not connect to broker URL:
tcp://localhost:61616. Reason: java.net.ConnectException: Connection
refused (Connection refused)
I usually get this same error even with the MariaDB setup. I know its
related to the Message broker but I don't know how to resolve it or how to
avoid it. Can anyone share their experience on this issue?
Also, I dropped the playground and seshat keyspaces in Cassandra as well as
in Postgres.
I changed the max-connection to 120 in PostgresSQL and also restarted the
system.
I am convinced the problem is not with PostgreSQL or Cassandra, plus, it
worked a few hours ago when I started it in lite mode.
PS: In light of this, how soon will this be merged into the develop
branches. I would love to completely switch to PostgreSQL. Toggling between
MariaDB and PostgreSQL is not easy :)
*At your service,*
*Graham.*
On Thu, 9 May 2019 at 20:13, Isaac Kamga <is...@mifos.org> wrote:
> Hello Dev,
>
> Thanks very much for testing Graham
>
> I've raised pull requests for the Migration to PostgreSQL. Demo-server
> runs to completion successfully in both cases and databases and tables are
> rightly being created.
> I had to make some adjustments to default-setup's ledgers.csv file because
> of an error that came from the CSVParser.
>
> *Testing demo-server lite for minimal set of services*
> I have fixed the issue with connecting to the right databases. You can use
> the postgres-setup-lite.sh
> <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh>
> script to test the minimal set of libraries and services. After the run,
> you should be able to ascertain that databases are created and tables are
> created where they have to be.
>
> +Juhan Aasaru <aa...@gmail.com> please provide feedback how this test
> goes.
>
> *Testing demo-server for all services*
> Given that the connection issues have been resolved, it's time to do more
> testing;
> @Juhan, @Graham and anyone else, in order to test the migration in it's
> entirety using postgres-setup.sh script
> <https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup.sh>,
> ensure to;
>
> 1. Install PostgreSQL
> 2. Change password for user postgres to postgres
> 3. Change max_connections to 120 in your postgresql.conf file ( usually in
> /etc/postgresql/... ) and restart the Postgresql service.Many connections
> to databases happening since all services are started.
> 4. Ensure Cassandra has no keyspaces seshat and playground
> 5. cd into a completely new directory from your usual Fineract CN
> development pwd
> 5. In integration-tests/fineract-cn-demo-server/build/libs/ and Run
> demo-server jar using *java -Ddemoserver.provision=true
> -Ddemoserver.persistent=true
> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
> <http://127.0.0.1:9042/> -Dcassandra.cluster.user=cassandra
> -Dcassandra.cluster.pwd=password -Dcustom.postgresql.host=localhost
> -Dcustom.postgresql.user=postgres -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar*
>
> Kindly provide feedback on how the run goes in the both cases.
>
> At Your Service,
> Isaac Kamga.
>
> On Fri, May 3, 2019 at 7:21 PM Ebenezer Graham <eg...@alustudent.com>
> wrote:
>
>> Hello Isaac / Juhan
>>
>> I missed a number of relevant steps and this led to the initial failure.
>>
>> The test has been successful and I will list the conditions which led to
>> the successful run below.
>>
>> 1. I created a new directory and cloned the repositories using the script
>> you provided (
>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>> )
>>
>> 2. I installed PostgreSQL server and changed the postgres user's password
>> to "postgres"
>>
>> 3. I used the command below to start the demo-server.
>>
>> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>> -Ddemoserver.lite=true -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>> -Dcustom.postgresql.host=localhost -Dcustom.postgresql.user=postgres -jar
>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>
>> @Juhan, please try this approach and let's know the outcome.
>>
>> You should get an output similar to this.
>>
>> 18:11:28.717 [Thread-16] INFO o.e.jetty.server.AbstractConnector -
>> Stopped ServerConnector@59a94d0f{HTTP/1.1,[http/1.1]}{0.0.0.0:2020}
>> 18:11:28.720 [Thread-16] INFO o.e.j.server.handler.ContextHandler -
>> Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@1213ffbc
>> {/provisioner/v1,file:///tmp/jetty-docbase.8613597725021372791.2020/,UNAVAILABLE}
>> identity-v1 address:http://localhost:2021/identity/v1
>> office-v1 address:http://localhost:2023/office/v1
>> customer-v1 address:http://localhost:2024/customer/v1
>>
>> At your service
>>
>>
>> On Fri, 3 May 2019 at 21:09, Ebenezer Graham <eg...@alustudent.com>
>> wrote:
>>
>>> Hello Isaac/Juhan,
>>>
>>> I have tried testing the code in lite mode for 3 times and I got the error
>>> below.
>>> https://pastebin.com/TaJqSNjQ
>>>
>>> Before the failure, I got this info.
>>>
>>> 16:56:03.519 [qtp1262607708-22] INFO o.a.f.c.l.c.ServiceExceptionFilter
>>> - Responding with a service error ServiceError{code=409, message='The given
>>> identity instance didnt recognize the system token as valid. Perhaps the
>>> system keys for the provisioner or for the identity manager are
>>> misconfigured?'}
>>> 16:57:03.620 [qtp1262607708-20] INFO o.a.f.c.l.c.ServiceExceptionFilter
>>> - Responding with a service error ServiceError{code=409, message='The given
>>> identity instance didnt recognize the system token as valid. Perhaps the
>>> system keys for the provisioner or for the identity manager are
>>> misconfigured?'}
>>>
>>> I will keep testing to get more insights into the error.
>>>
>>> *At your service,*
>>>
>>> *Ebenezer Graham*
>>>
>>> *BSc (Hons) Computing*
>>>
>>>
>>> [image: EmailSignature.png]
>>>
>>> African Leadership University,
>>>
>>> Power Mill Road, Pamplemousses,
>>>
>>> Mauritius.
>>> GitHub <https://github.com/ebenezergraham> | LinkedIn
>>> <https://www.linkedin.com/in/ebenezer-graham/> | ebenezergraham.me
>>> *Skype*:
>>> ebenezer.graham <http://ebenezergraham.me/>
>>>
>>> *This email and any files transmitted with it are confidential and
>>> intended solely for the use of the individual or entity to whom they are
>>> addressed. If you have received this email in error please notify the
>>> sender and then delete this email. If you are not the intended recipient
>>> you are notified that disclosing, copying, distributing or taking any
>>> action in reliance on the contents of this information is strictly
>>> prohibited.*
>>>
>>>
>>> On Thu, 2 May 2019 at 06:38, Isaac Kamga <is...@mifos.org> wrote:
>>>
>>>> Hello Juhan,
>>>>
>>>> I've updated the PRs with commits to fix the errors you were
>>>> experiencing.
>>>>
>>>> Could you kindly wipe Cassandra and PostgreSQL clean, test once more
>>>> and provide feedback on how it goes ?
>>>>
>>>> Cheers,
>>>> Isaac Kamga.
>>>>
>>>> On Tue, Apr 9, 2019 at 4:36 PM Isaac Kamga <is...@mifos.org>
>>>> wrote:
>>>>
>>>>> Hey Juhan,
>>>>>
>>>>> Thanks for testing.
>>>>>
>>>>> Wipe clean your Cassandra keyspaces first.
>>>>>
>>>>> I suspect that what's interferring is the keyspaces from your last run
>>>>> when using MariaDB.
>>>>>
>>>>> Please provide feedback how it goes.
>>>>>
>>>>> Cheers,
>>>>> Isaac Kamga.
>>>>>
>>>>> On Sun, Mar 31, 2019 at 10:00 AM Juhan Aasaru <aa...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I Isaac,
>>>>>>
>>>>>> I pulled the changes and rebuilt the projects.
>>>>>>
>>>>>> Now when I run (postgres and cassandra working locally):
>>>>>>
>>>>>> java -Ddemoserver.persistent=true -Ddemoserver.provision=true
>>>>>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>>>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>>>>> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>>
>>>>>> I get an error here:
>>>>>>
>>>>>> 11:18:42.066 [main] INFO o.s.b.c.e.j.JettyEmbeddedServletContainer -
>>>>>> Jetty
>>>>>> started on port(s) 2024 (http/1.1)
>>>>>> 11:18:42.067 [main] INFO
>>>>>> o.s.c.n.e.EurekaDiscoveryClientConfiguration -
>>>>>> Updating port to 2024
>>>>>> 11:18:42.072 [main] INFO o.a.f.c.customer.CustomerApplication -
>>>>>> Started
>>>>>> CustomerApplication in 16.423 seconds (JVM running for 17.342)
>>>>>> 11:18:57.435 [qtp1479696465-21] INFO
>>>>>> o.a.f.c.l.c.ServiceExceptionFilter -
>>>>>> Responding with a service error ServiceError{code=409,
>>>>>> message='Application
>>>>>> identity-v1 already exists!'}
>>>>>> 11:18:57.464 [Thread-16] INFO
>>>>>> o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing
>>>>>>
>>>>>> org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@4d50efb8
>>>>>> :
>>>>>> startup date [Sun Mar 31 11:12:52 EEST 2019]; parent:
>>>>>>
>>>>>> org.springframework.context.annotation.AnnotationConfigApplicationContext@75a1cd57
>>>>>>
>>>>>> I understand I have to use -Ddemoserver.provision=true flag to
>>>>>> provision
>>>>>> the initial database so I won't be able to take that one out.
>>>>>>
>>>>>> Juhan
>>>>>>
>>>>>>
>>>>>> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval R,
>>>>>> 29.
>>>>>> märts 2019 kell 04:17:
>>>>>>
>>>>>> > Hey Juhan,
>>>>>> >
>>>>>> > I have updated the Pull request to fineract-cn-provisioner with a
>>>>>> commit
>>>>>> > [1] which fixes the errors you're experiencing.
>>>>>> >
>>>>>> > I'll advise that you test using option -Ddemoserver.persistent=true
>>>>>> and
>>>>>> > -Ddemoserver.lite=true and we're testing just the minimal set of
>>>>>> services
>>>>>> > now. In fact,I tested and the tail of my logs showed thus
>>>>>> > *identity-v1 address:http://localhost:2021/identity/v1
>>>>>> > <
>>>>>> >
>>>>>> https://www.google.com/url?q=http://localhost:2021/identity/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNEjCID96mw3LF1ZIPmjEIQkHh_LcQ
>>>>>> > >*
>>>>>> >
>>>>>> > *office-v1 address:http://localhost:2023/office/v1
>>>>>> > <
>>>>>> >
>>>>>> https://www.google.com/url?q=http://localhost:2023/office/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNGjyk1BHroLddOTd0FhJUJG4zdVag
>>>>>> > >customer-v1
>>>>>> > address:http://localhost:2024/customer/v1
>>>>>> > <
>>>>>> >
>>>>>> https://www.google.com/url?q=http://localhost:2024/customer/v1&sa=D&source=hangouts&ust=1553910362599000&usg=AFQjCNGfcoJZIxmcsX34YeEXRUzS0oC2hA
>>>>>> > >*
>>>>>> > Albeit, this error [2] turned up related to the Notifications
>>>>>> service. Just
>>>>>> > ignore it for now as it will be handled down the line during
>>>>>> extensive
>>>>>> > integration testing.
>>>>>> >
>>>>>> > We can keep testing using -Ddemoserver.provision=true for later too
>>>>>> when
>>>>>> > we've migrated and tested the individual services.
>>>>>> >
>>>>>> > Can you kindly update your provisioner repository and test once
>>>>>> more and
>>>>>> > let's keep fixing ?
>>>>>> >
>>>>>> > [1]
>>>>>> >
>>>>>> >
>>>>>> https://github.com/Izakey/fineract-cn-provisioner/commit/b83331b723f175003ccf069764414e4ab371662c
>>>>>> >
>>>>>> > [2] https://pastebin.com/DBzNyHir
>>>>>> >
>>>>>> > Cheers,
>>>>>> > Isaac Kamga.
>>>>>> >
>>>>>> > On Wed, Mar 27, 2019 at 10:23 PM Isaac Kamga <isaac.kamga@mifos.org
>>>>>> >
>>>>>> > wrote:
>>>>>> >
>>>>>> > > Hello Juhan,
>>>>>> > >
>>>>>> > > Thank you for taking the time to do some testing.
>>>>>> > >
>>>>>> > > On Wed, Mar 27, 2019 at 10:56 AM Juhan Aasaru <aa...@gmail.com>
>>>>>> wrote:
>>>>>> > >
>>>>>> > >> Hi Isaac,
>>>>>> > >>
>>>>>> > >> great initiative towards pushing to PostgreSQL.
>>>>>> > >>
>>>>>> > >> I looked at your instructions and tried to run your changes
>>>>>> locally.
>>>>>> > >> The sh script came in handy to quickly clone and build
>>>>>> everything that
>>>>>> > was
>>>>>> > >> needed.
>>>>>> > >> But then I got confused about provisioning.
>>>>>> > >>
>>>>>> > >> Question #1: How exactly I should run demo-server? I tried the
>>>>>> following
>>>>>> > >> options:
>>>>>> > >>
>>>>>> > >> 1. I didn't have postgre and cassandra running locally.
>>>>>> > >> java -jar -Ddemoserver.lite=true
>>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>> > >>
>>>>>> > >> resulted
>>>>>> > >>
>>>>>> > >> org.springframework.beans.factory.BeanCreationException: Error
>>>>>> creating
>>>>>> > >> bean with name 'provisionerInitializer':
>>>>>> > >> Invocation of init method failed; nested exception is
>>>>>> > >> java.lang.IllegalStateException: Could not initialize service!
>>>>>> > >> Caused by: org.postgresql.util.PSQLException: FATAL: database
>>>>>> "seshat"
>>>>>> > >> does
>>>>>> > >> not exist
>>>>>> > >>
>>>>>> > >
>>>>>> > > I haven't done testing using this option yet.
>>>>>> > >
>>>>>> > >
>>>>>> > >>
>>>>>> > >> 2. I started posgresql (and created a database seshat) and
>>>>>> cassandra and
>>>>>> > >> ran:
>>>>>> > >> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>>>>>> > >> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 \
>>>>>> > >> -Dcassandra.cluster.user=cassandra
>>>>>> -Dcassandra.cluster.pwd=password
>>>>>> > >> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>> > >>
>>>>>> > >> Initially applications started up one after anther
>>>>>> > >> (ProvisionerApplication,
>>>>>> > >> RhythmApplication, OfficeApplication)
>>>>>> > >> but CustomerApplication threw an exception:
>>>>>> > >>
>>>>>> > >> 11:50:39.377 [qtp1424482154-21] WARN
>>>>>> o.e.jetty.servlet.ServletHandler -
>>>>>> > >> org.springframework.web.util.NestedServletException: Request
>>>>>> processing
>>>>>> > >> failed; nested exception is java.lang.IllegalStateException:
>>>>>> Could not
>>>>>> > >> insert org.apache.fineract.cn.provisioner.tenant info!
>>>>>> > >> Caused by: org.postgresql.util.PSQLException: ERROR: relation
>>>>>> > >> "seshat.tenants" does not exist
>>>>>> > >>
>>>>>> > >
>>>>>> > > Try running the same command above without the provision=true
>>>>>> option and
>>>>>> > > share what you get.
>>>>>> > >
>>>>>> > > I had to create database seshat in postgresql and the can't find
>>>>>> database
>>>>>> > > error went away. Down the road, also encountered the same error
>>>>>> > > <https://pastebin.com/qCT1T5KM> "Could not insert ....
>>>>>> > provisioner.tenant
>>>>>> > > info !"
>>>>>> > > I'm currently working on resolving this.
>>>>>> > >
>>>>>> > >
>>>>>> > >>
>>>>>> > >> Question #2 - do I still need to have Mysql running? I see that
>>>>>> you
>>>>>> > >> haven't
>>>>>> > >> made code changes to identity
>>>>>> > >> which as I understand still uses Mysql.
>>>>>> > >>
>>>>>> > >
>>>>>> > > I don't think you need MySQL running to test the migrations
>>>>>> currently
>>>>>> > > underway and there was no need for code changes in the identity
>>>>>> service.
>>>>>> > > However, it could be helpful to see the structure of the
>>>>>> databases in
>>>>>> > MySQL
>>>>>> > > so that it helps with debugging and testing.
>>>>>> > >
>>>>>> > > Whenever you're available, tell me so we schedule a call so I
>>>>>> look into
>>>>>> > > what your experiencing too.
>>>>>> > >
>>>>>> > > Cheers,
>>>>>> > > Isaac Kamga.
>>>>>> > >
>>>>>> > >
>>>>>> > >>
>>>>>> > >> Kind regards
>>>>>> > >> Juhan
>>>>>> > >>
>>>>>> > >>
>>>>>> > >> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas
>>>>>> kuupäeval L, 23.
>>>>>> > >> märts 2019 kell 04:31:
>>>>>> > >>
>>>>>> > >> > Hello fineracters,
>>>>>> > >> >
>>>>>> > >> > Trust that you had a great week.
>>>>>> > >> >
>>>>>> > >> > I just finished migrating libraries PostgreSQL[1] and Test[2]
>>>>>> from
>>>>>> > >> MariaDB
>>>>>> > >> > to PostgreSQL, mostly adapting SQL statements & database
>>>>>> connections
>>>>>> > to
>>>>>> > >> > work with PostgreSQL and reverting back to Hibernate ORM until
>>>>>> > database
>>>>>> > >> > migrations are completed.
>>>>>> > >> >
>>>>>> > >> > I have also migrated the "minimal" services viz provisioner[3],
>>>>>> > >> rhythm[4],
>>>>>> > >> > office[5], template[6] and customer[7] from MariaDB to
>>>>>> PostgreSQL. The
>>>>>> > >> > Spring datasource credentials had to be incorporated into
>>>>>> > >> application.yaml
>>>>>> > >> > files to make things work too.
>>>>>> > >> >
>>>>>> > >> > Given I updated service-starter [8] and demo-server [9],
>>>>>> developers
>>>>>> > can
>>>>>> > >> > test the migrated services when running the
>>>>>> *-Ddemoserver.lite=true*
>>>>>> > >> option
>>>>>> > >> > which builds the minimal set of services.
>>>>>> > >> >
>>>>>> > >> > You'll observe that some of the pull requests are heading to
>>>>>> develop
>>>>>> > >> > branches ( It appears comparing changes and creating PRs
>>>>>> towards the
>>>>>> > >> same
>>>>>> > >> > branch name from origin to upstream expires after a few
>>>>>> minutes on
>>>>>> > >> > Github... weird )
>>>>>> > >> >
>>>>>> > >> > Here is a script [10] to enable anyone clone and do an
>>>>>> > integration-test
>>>>>> > >> of
>>>>>> > >> > the new changes using demo-server.
>>>>>> > >> >
>>>>>> > >> > Kindly test these changes and provide feedback.
>>>>>> > >> >
>>>>>> > >> > At Your Service,
>>>>>> > >> > Isaac Kamga.
>>>>>> > >> >
>>>>>> > >> > [1] https://github.com/apache/fineract-cn-postgresql/pull/1
>>>>>> > >> > [2] https://github.com/apache/fineract-cn-test/pull/7
>>>>>> > >> > [3] https://github.com/apache/fineract-cn-provisioner/pull/6
>>>>>> > >> > [4] https://github.com/apache/fineract-cn-rhythm/pull/8
>>>>>> > >> > [5] https://github.com/apache/fineract-cn-office/pull/9
>>>>>> > >> > [6] https://github.com/apache/fineract-cn-template/pull/6
>>>>>> > >> > [7] https://github.com/apache/fineract-cn-customer/pull/10
>>>>>> > >> > [8]
>>>>>> https://github.com/apache/fineract-cn-service-starter/pull/8
>>>>>> > >> > [9] https://github.com/apache/fineract-cn-demo-server/pull/27
>>>>>> > >> > [10]
>>>>>> > >> >
>>>>>> > >> >
>>>>>> > >>
>>>>>> >
>>>>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>>>>>> > >> >
>>>>>> > >>
>>>>>> > >
>>>>>> >
>>>>>>
>>>>>
Re: updates On Migration from MariaDB to PostgreSQL
Posted by Isaac Kamga <is...@mifos.org>.
Hello Dev,
Thanks very much for testing Graham
I've raised pull requests for the Migration to PostgreSQL. Demo-server runs
to completion successfully in both cases and databases and tables are
rightly being created.
I had to make some adjustments to default-setup's ledgers.csv file because
of an error that came from the CSVParser.
*Testing demo-server lite for minimal set of services*
I have fixed the issue with connecting to the right databases. You can use
the postgres-setup-lite.sh
<https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh>
script to test the minimal set of libraries and services. After the run,
you should be able to ascertain that databases are created and tables are
created where they have to be.
+Juhan Aasaru <aa...@gmail.com> please provide feedback how this test goes.
*Testing demo-server for all services*
Given that the connection issues have been resolved, it's time to do more
testing;
@Juhan, @Graham and anyone else, in order to test the migration in it's
entirety using postgres-setup.sh script
<https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup.sh>,
ensure to;
1. Install PostgreSQL
2. Change password for user postgres to postgres
3. Change max_connections to 120 in your postgresql.conf file ( usually in
/etc/postgresql/... ) and restart the Postgresql service.Many connections
to databases happening since all services are started.
4. Ensure Cassandra has no keyspaces seshat and playground
5. cd into a completely new directory from your usual Fineract CN
development pwd
5. In integration-tests/fineract-cn-demo-server/build/libs/ and Run
demo-server jar using *java -Ddemoserver.provision=true
-Ddemoserver.persistent=true
-Dcustom.cassandra.contactPoints=127.0.0.1:9042
<http://127.0.0.1:9042/> -Dcassandra.cluster.user=cassandra
-Dcassandra.cluster.pwd=password -Dcustom.postgresql.host=localhost
-Dcustom.postgresql.user=postgres -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar*
Kindly provide feedback on how the run goes in the both cases.
At Your Service,
Isaac Kamga.
On Fri, May 3, 2019 at 7:21 PM Ebenezer Graham <eg...@alustudent.com>
wrote:
> Hello Isaac / Juhan
>
> I missed a number of relevant steps and this led to the initial failure.
>
> The test has been successful and I will list the conditions which led to
> the successful run below.
>
> 1. I created a new directory and cloned the repositories using the script
> you provided (
> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
> )
>
> 2. I installed PostgreSQL server and changed the postgres user's password
> to "postgres"
>
> 3. I used the command below to start the demo-server.
>
> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
> -Ddemoserver.lite=true -Dcustom.cassandra.contactPoints=127.0.0.1:9042
> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
> -Dcustom.postgresql.host=localhost -Dcustom.postgresql.user=postgres -jar
> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>
> @Juhan, please try this approach and let's know the outcome.
>
> You should get an output similar to this.
>
> 18:11:28.717 [Thread-16] INFO o.e.jetty.server.AbstractConnector -
> Stopped ServerConnector@59a94d0f{HTTP/1.1,[http/1.1]}{0.0.0.0:2020}
> 18:11:28.720 [Thread-16] INFO o.e.j.server.handler.ContextHandler -
> Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@1213ffbc
> {/provisioner/v1,file:///tmp/jetty-docbase.8613597725021372791.2020/,UNAVAILABLE}
> identity-v1 address:http://localhost:2021/identity/v1
> office-v1 address:http://localhost:2023/office/v1
> customer-v1 address:http://localhost:2024/customer/v1
>
> At your service
>
>
> On Fri, 3 May 2019 at 21:09, Ebenezer Graham <eg...@alustudent.com>
> wrote:
>
>> Hello Isaac/Juhan,
>>
>> I have tried testing the code in lite mode for 3 times and I got the error
>> below.
>> https://pastebin.com/TaJqSNjQ
>>
>> Before the failure, I got this info.
>>
>> 16:56:03.519 [qtp1262607708-22] INFO o.a.f.c.l.c.ServiceExceptionFilter
>> - Responding with a service error ServiceError{code=409, message='The given
>> identity instance didnt recognize the system token as valid. Perhaps the
>> system keys for the provisioner or for the identity manager are
>> misconfigured?'}
>> 16:57:03.620 [qtp1262607708-20] INFO o.a.f.c.l.c.ServiceExceptionFilter
>> - Responding with a service error ServiceError{code=409, message='The given
>> identity instance didnt recognize the system token as valid. Perhaps the
>> system keys for the provisioner or for the identity manager are
>> misconfigured?'}
>>
>> I will keep testing to get more insights into the error.
>>
>> *At your service,*
>>
>> *Ebenezer Graham*
>>
>> *BSc (Hons) Computing*
>>
>>
>> [image: EmailSignature.png]
>>
>> African Leadership University,
>>
>> Power Mill Road, Pamplemousses,
>>
>> Mauritius.
>> GitHub <https://github.com/ebenezergraham> | LinkedIn
>> <https://www.linkedin.com/in/ebenezer-graham/> | ebenezergraham.me
>> *Skype*:
>> ebenezer.graham <http://ebenezergraham.me/>
>>
>> *This email and any files transmitted with it are confidential and
>> intended solely for the use of the individual or entity to whom they are
>> addressed. If you have received this email in error please notify the
>> sender and then delete this email. If you are not the intended recipient
>> you are notified that disclosing, copying, distributing or taking any
>> action in reliance on the contents of this information is strictly
>> prohibited.*
>>
>>
>> On Thu, 2 May 2019 at 06:38, Isaac Kamga <is...@mifos.org> wrote:
>>
>>> Hello Juhan,
>>>
>>> I've updated the PRs with commits to fix the errors you were
>>> experiencing.
>>>
>>> Could you kindly wipe Cassandra and PostgreSQL clean, test once more and
>>> provide feedback on how it goes ?
>>>
>>> Cheers,
>>> Isaac Kamga.
>>>
>>> On Tue, Apr 9, 2019 at 4:36 PM Isaac Kamga <is...@mifos.org>
>>> wrote:
>>>
>>>> Hey Juhan,
>>>>
>>>> Thanks for testing.
>>>>
>>>> Wipe clean your Cassandra keyspaces first.
>>>>
>>>> I suspect that what's interferring is the keyspaces from your last run
>>>> when using MariaDB.
>>>>
>>>> Please provide feedback how it goes.
>>>>
>>>> Cheers,
>>>> Isaac Kamga.
>>>>
>>>> On Sun, Mar 31, 2019 at 10:00 AM Juhan Aasaru <aa...@gmail.com> wrote:
>>>>
>>>>> I Isaac,
>>>>>
>>>>> I pulled the changes and rebuilt the projects.
>>>>>
>>>>> Now when I run (postgres and cassandra working locally):
>>>>>
>>>>> java -Ddemoserver.persistent=true -Ddemoserver.provision=true
>>>>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>>>> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>>
>>>>> I get an error here:
>>>>>
>>>>> 11:18:42.066 [main] INFO o.s.b.c.e.j.JettyEmbeddedServletContainer -
>>>>> Jetty
>>>>> started on port(s) 2024 (http/1.1)
>>>>> 11:18:42.067 [main] INFO o.s.c.n.e.EurekaDiscoveryClientConfiguration
>>>>> -
>>>>> Updating port to 2024
>>>>> 11:18:42.072 [main] INFO o.a.f.c.customer.CustomerApplication -
>>>>> Started
>>>>> CustomerApplication in 16.423 seconds (JVM running for 17.342)
>>>>> 11:18:57.435 [qtp1479696465-21] INFO
>>>>> o.a.f.c.l.c.ServiceExceptionFilter -
>>>>> Responding with a service error ServiceError{code=409,
>>>>> message='Application
>>>>> identity-v1 already exists!'}
>>>>> 11:18:57.464 [Thread-16] INFO
>>>>> o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing
>>>>>
>>>>> org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@4d50efb8
>>>>> :
>>>>> startup date [Sun Mar 31 11:12:52 EEST 2019]; parent:
>>>>>
>>>>> org.springframework.context.annotation.AnnotationConfigApplicationContext@75a1cd57
>>>>>
>>>>> I understand I have to use -Ddemoserver.provision=true flag to
>>>>> provision
>>>>> the initial database so I won't be able to take that one out.
>>>>>
>>>>> Juhan
>>>>>
>>>>>
>>>>> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval R,
>>>>> 29.
>>>>> märts 2019 kell 04:17:
>>>>>
>>>>> > Hey Juhan,
>>>>> >
>>>>> > I have updated the Pull request to fineract-cn-provisioner with a
>>>>> commit
>>>>> > [1] which fixes the errors you're experiencing.
>>>>> >
>>>>> > I'll advise that you test using option -Ddemoserver.persistent=true
>>>>> and
>>>>> > -Ddemoserver.lite=true and we're testing just the minimal set of
>>>>> services
>>>>> > now. In fact,I tested and the tail of my logs showed thus
>>>>> > *identity-v1 address:http://localhost:2021/identity/v1
>>>>> > <
>>>>> >
>>>>> https://www.google.com/url?q=http://localhost:2021/identity/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNEjCID96mw3LF1ZIPmjEIQkHh_LcQ
>>>>> > >*
>>>>> >
>>>>> > *office-v1 address:http://localhost:2023/office/v1
>>>>> > <
>>>>> >
>>>>> https://www.google.com/url?q=http://localhost:2023/office/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNGjyk1BHroLddOTd0FhJUJG4zdVag
>>>>> > >customer-v1
>>>>> > address:http://localhost:2024/customer/v1
>>>>> > <
>>>>> >
>>>>> https://www.google.com/url?q=http://localhost:2024/customer/v1&sa=D&source=hangouts&ust=1553910362599000&usg=AFQjCNGfcoJZIxmcsX34YeEXRUzS0oC2hA
>>>>> > >*
>>>>> > Albeit, this error [2] turned up related to the Notifications
>>>>> service. Just
>>>>> > ignore it for now as it will be handled down the line during
>>>>> extensive
>>>>> > integration testing.
>>>>> >
>>>>> > We can keep testing using -Ddemoserver.provision=true for later too
>>>>> when
>>>>> > we've migrated and tested the individual services.
>>>>> >
>>>>> > Can you kindly update your provisioner repository and test once more
>>>>> and
>>>>> > let's keep fixing ?
>>>>> >
>>>>> > [1]
>>>>> >
>>>>> >
>>>>> https://github.com/Izakey/fineract-cn-provisioner/commit/b83331b723f175003ccf069764414e4ab371662c
>>>>> >
>>>>> > [2] https://pastebin.com/DBzNyHir
>>>>> >
>>>>> > Cheers,
>>>>> > Isaac Kamga.
>>>>> >
>>>>> > On Wed, Mar 27, 2019 at 10:23 PM Isaac Kamga <is...@mifos.org>
>>>>> > wrote:
>>>>> >
>>>>> > > Hello Juhan,
>>>>> > >
>>>>> > > Thank you for taking the time to do some testing.
>>>>> > >
>>>>> > > On Wed, Mar 27, 2019 at 10:56 AM Juhan Aasaru <aa...@gmail.com>
>>>>> wrote:
>>>>> > >
>>>>> > >> Hi Isaac,
>>>>> > >>
>>>>> > >> great initiative towards pushing to PostgreSQL.
>>>>> > >>
>>>>> > >> I looked at your instructions and tried to run your changes
>>>>> locally.
>>>>> > >> The sh script came in handy to quickly clone and build everything
>>>>> that
>>>>> > was
>>>>> > >> needed.
>>>>> > >> But then I got confused about provisioning.
>>>>> > >>
>>>>> > >> Question #1: How exactly I should run demo-server? I tried the
>>>>> following
>>>>> > >> options:
>>>>> > >>
>>>>> > >> 1. I didn't have postgre and cassandra running locally.
>>>>> > >> java -jar -Ddemoserver.lite=true
>>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>> > >>
>>>>> > >> resulted
>>>>> > >>
>>>>> > >> org.springframework.beans.factory.BeanCreationException: Error
>>>>> creating
>>>>> > >> bean with name 'provisionerInitializer':
>>>>> > >> Invocation of init method failed; nested exception is
>>>>> > >> java.lang.IllegalStateException: Could not initialize service!
>>>>> > >> Caused by: org.postgresql.util.PSQLException: FATAL: database
>>>>> "seshat"
>>>>> > >> does
>>>>> > >> not exist
>>>>> > >>
>>>>> > >
>>>>> > > I haven't done testing using this option yet.
>>>>> > >
>>>>> > >
>>>>> > >>
>>>>> > >> 2. I started posgresql (and created a database seshat) and
>>>>> cassandra and
>>>>> > >> ran:
>>>>> > >> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>>>>> > >> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 \
>>>>> > >> -Dcassandra.cluster.user=cassandra
>>>>> -Dcassandra.cluster.pwd=password
>>>>> > >> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>> > >>
>>>>> > >> Initially applications started up one after anther
>>>>> > >> (ProvisionerApplication,
>>>>> > >> RhythmApplication, OfficeApplication)
>>>>> > >> but CustomerApplication threw an exception:
>>>>> > >>
>>>>> > >> 11:50:39.377 [qtp1424482154-21] WARN
>>>>> o.e.jetty.servlet.ServletHandler -
>>>>> > >> org.springframework.web.util.NestedServletException: Request
>>>>> processing
>>>>> > >> failed; nested exception is java.lang.IllegalStateException:
>>>>> Could not
>>>>> > >> insert org.apache.fineract.cn.provisioner.tenant info!
>>>>> > >> Caused by: org.postgresql.util.PSQLException: ERROR: relation
>>>>> > >> "seshat.tenants" does not exist
>>>>> > >>
>>>>> > >
>>>>> > > Try running the same command above without the provision=true
>>>>> option and
>>>>> > > share what you get.
>>>>> > >
>>>>> > > I had to create database seshat in postgresql and the can't find
>>>>> database
>>>>> > > error went away. Down the road, also encountered the same error
>>>>> > > <https://pastebin.com/qCT1T5KM> "Could not insert ....
>>>>> > provisioner.tenant
>>>>> > > info !"
>>>>> > > I'm currently working on resolving this.
>>>>> > >
>>>>> > >
>>>>> > >>
>>>>> > >> Question #2 - do I still need to have Mysql running? I see that
>>>>> you
>>>>> > >> haven't
>>>>> > >> made code changes to identity
>>>>> > >> which as I understand still uses Mysql.
>>>>> > >>
>>>>> > >
>>>>> > > I don't think you need MySQL running to test the migrations
>>>>> currently
>>>>> > > underway and there was no need for code changes in the identity
>>>>> service.
>>>>> > > However, it could be helpful to see the structure of the databases
>>>>> in
>>>>> > MySQL
>>>>> > > so that it helps with debugging and testing.
>>>>> > >
>>>>> > > Whenever you're available, tell me so we schedule a call so I look
>>>>> into
>>>>> > > what your experiencing too.
>>>>> > >
>>>>> > > Cheers,
>>>>> > > Isaac Kamga.
>>>>> > >
>>>>> > >
>>>>> > >>
>>>>> > >> Kind regards
>>>>> > >> Juhan
>>>>> > >>
>>>>> > >>
>>>>> > >> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval
>>>>> L, 23.
>>>>> > >> märts 2019 kell 04:31:
>>>>> > >>
>>>>> > >> > Hello fineracters,
>>>>> > >> >
>>>>> > >> > Trust that you had a great week.
>>>>> > >> >
>>>>> > >> > I just finished migrating libraries PostgreSQL[1] and Test[2]
>>>>> from
>>>>> > >> MariaDB
>>>>> > >> > to PostgreSQL, mostly adapting SQL statements & database
>>>>> connections
>>>>> > to
>>>>> > >> > work with PostgreSQL and reverting back to Hibernate ORM until
>>>>> > database
>>>>> > >> > migrations are completed.
>>>>> > >> >
>>>>> > >> > I have also migrated the "minimal" services viz provisioner[3],
>>>>> > >> rhythm[4],
>>>>> > >> > office[5], template[6] and customer[7] from MariaDB to
>>>>> PostgreSQL. The
>>>>> > >> > Spring datasource credentials had to be incorporated into
>>>>> > >> application.yaml
>>>>> > >> > files to make things work too.
>>>>> > >> >
>>>>> > >> > Given I updated service-starter [8] and demo-server [9],
>>>>> developers
>>>>> > can
>>>>> > >> > test the migrated services when running the
>>>>> *-Ddemoserver.lite=true*
>>>>> > >> option
>>>>> > >> > which builds the minimal set of services.
>>>>> > >> >
>>>>> > >> > You'll observe that some of the pull requests are heading to
>>>>> develop
>>>>> > >> > branches ( It appears comparing changes and creating PRs
>>>>> towards the
>>>>> > >> same
>>>>> > >> > branch name from origin to upstream expires after a few minutes
>>>>> on
>>>>> > >> > Github... weird )
>>>>> > >> >
>>>>> > >> > Here is a script [10] to enable anyone clone and do an
>>>>> > integration-test
>>>>> > >> of
>>>>> > >> > the new changes using demo-server.
>>>>> > >> >
>>>>> > >> > Kindly test these changes and provide feedback.
>>>>> > >> >
>>>>> > >> > At Your Service,
>>>>> > >> > Isaac Kamga.
>>>>> > >> >
>>>>> > >> > [1] https://github.com/apache/fineract-cn-postgresql/pull/1
>>>>> > >> > [2] https://github.com/apache/fineract-cn-test/pull/7
>>>>> > >> > [3] https://github.com/apache/fineract-cn-provisioner/pull/6
>>>>> > >> > [4] https://github.com/apache/fineract-cn-rhythm/pull/8
>>>>> > >> > [5] https://github.com/apache/fineract-cn-office/pull/9
>>>>> > >> > [6] https://github.com/apache/fineract-cn-template/pull/6
>>>>> > >> > [7] https://github.com/apache/fineract-cn-customer/pull/10
>>>>> > >> > [8]
>>>>> https://github.com/apache/fineract-cn-service-starter/pull/8
>>>>> > >> > [9] https://github.com/apache/fineract-cn-demo-server/pull/27
>>>>> > >> > [10]
>>>>> > >> >
>>>>> > >> >
>>>>> > >>
>>>>> >
>>>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>>>>> > >> >
>>>>> > >>
>>>>> > >
>>>>> >
>>>>>
>>>>
Re: updates On Migration from MariaDB to PostgreSQL
Posted by Ebenezer Graham <eg...@alustudent.com>.
Hello Isaac / Juhan
I missed a number of relevant steps and this led to the initial failure.
The test has been successful and I will list the conditions which led to
the successful run below.
1. I created a new directory and cloned the repositories using the script
you provided (
https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
)
2. I installed PostgreSQL server and changed the postgres user's password
to "postgres"
3. I used the command below to start the demo-server.
java -Ddemoserver.provision=true -Ddemoserver.persistent=true
-Ddemoserver.lite=true -Dcustom.cassandra.contactPoints=127.0.0.1:9042
-Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
-Dcustom.postgresql.host=localhost -Dcustom.postgresql.user=postgres -jar
demo-server-0.1.0-BUILD-SNAPSHOT.jar
@Juhan, please try this approach and let's know the outcome.
You should get an output similar to this.
18:11:28.717 [Thread-16] INFO o.e.jetty.server.AbstractConnector - Stopped
ServerConnector@59a94d0f{HTTP/1.1,[http/1.1]}{0.0.0.0:2020}
18:11:28.720 [Thread-16] INFO o.e.j.server.handler.ContextHandler -
Stopped o.s.b.c.e.j.JettyEmbeddedWebAppContext@1213ffbc
{/provisioner/v1,file:///tmp/jetty-docbase.8613597725021372791.2020/,UNAVAILABLE}
identity-v1 address:http://localhost:2021/identity/v1
office-v1 address:http://localhost:2023/office/v1
customer-v1 address:http://localhost:2024/customer/v1
At your service
On Fri, 3 May 2019 at 21:09, Ebenezer Graham <eg...@alustudent.com>
wrote:
> Hello Isaac/Juhan,
>
> I have tried testing the code in lite mode for 3 times and I got the error
> below.
> https://pastebin.com/TaJqSNjQ
>
> Before the failure, I got this info.
>
> 16:56:03.519 [qtp1262607708-22] INFO o.a.f.c.l.c.ServiceExceptionFilter -
> Responding with a service error ServiceError{code=409, message='The given
> identity instance didnt recognize the system token as valid. Perhaps the
> system keys for the provisioner or for the identity manager are
> misconfigured?'}
> 16:57:03.620 [qtp1262607708-20] INFO o.a.f.c.l.c.ServiceExceptionFilter -
> Responding with a service error ServiceError{code=409, message='The given
> identity instance didnt recognize the system token as valid. Perhaps the
> system keys for the provisioner or for the identity manager are
> misconfigured?'}
>
> I will keep testing to get more insights into the error.
>
> *At your service,*
>
> *Ebenezer Graham*
>
> *BSc (Hons) Computing*
>
>
> [image: EmailSignature.png]
>
> African Leadership University,
>
> Power Mill Road, Pamplemousses,
>
> Mauritius.
> GitHub <https://github.com/ebenezergraham> | LinkedIn
> <https://www.linkedin.com/in/ebenezer-graham/> | ebenezergraham.me
> *Skype*:
> ebenezer.graham <http://ebenezergraham.me/>
>
> *This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they are
> addressed. If you have received this email in error please notify the
> sender and then delete this email. If you are not the intended recipient
> you are notified that disclosing, copying, distributing or taking any
> action in reliance on the contents of this information is strictly
> prohibited.*
>
>
> On Thu, 2 May 2019 at 06:38, Isaac Kamga <is...@mifos.org> wrote:
>
>> Hello Juhan,
>>
>> I've updated the PRs with commits to fix the errors you were experiencing.
>>
>> Could you kindly wipe Cassandra and PostgreSQL clean, test once more and
>> provide feedback on how it goes ?
>>
>> Cheers,
>> Isaac Kamga.
>>
>> On Tue, Apr 9, 2019 at 4:36 PM Isaac Kamga <is...@mifos.org> wrote:
>>
>>> Hey Juhan,
>>>
>>> Thanks for testing.
>>>
>>> Wipe clean your Cassandra keyspaces first.
>>>
>>> I suspect that what's interferring is the keyspaces from your last run
>>> when using MariaDB.
>>>
>>> Please provide feedback how it goes.
>>>
>>> Cheers,
>>> Isaac Kamga.
>>>
>>> On Sun, Mar 31, 2019 at 10:00 AM Juhan Aasaru <aa...@gmail.com> wrote:
>>>
>>>> I Isaac,
>>>>
>>>> I pulled the changes and rebuilt the projects.
>>>>
>>>> Now when I run (postgres and cassandra working locally):
>>>>
>>>> java -Ddemoserver.persistent=true -Ddemoserver.provision=true
>>>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>>> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>>
>>>> I get an error here:
>>>>
>>>> 11:18:42.066 [main] INFO o.s.b.c.e.j.JettyEmbeddedServletContainer -
>>>> Jetty
>>>> started on port(s) 2024 (http/1.1)
>>>> 11:18:42.067 [main] INFO o.s.c.n.e.EurekaDiscoveryClientConfiguration -
>>>> Updating port to 2024
>>>> 11:18:42.072 [main] INFO o.a.f.c.customer.CustomerApplication - Started
>>>> CustomerApplication in 16.423 seconds (JVM running for 17.342)
>>>> 11:18:57.435 [qtp1479696465-21] INFO
>>>> o.a.f.c.l.c.ServiceExceptionFilter -
>>>> Responding with a service error ServiceError{code=409,
>>>> message='Application
>>>> identity-v1 already exists!'}
>>>> 11:18:57.464 [Thread-16] INFO
>>>> o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing
>>>>
>>>> org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@4d50efb8
>>>> :
>>>> startup date [Sun Mar 31 11:12:52 EEST 2019]; parent:
>>>>
>>>> org.springframework.context.annotation.AnnotationConfigApplicationContext@75a1cd57
>>>>
>>>> I understand I have to use -Ddemoserver.provision=true flag to provision
>>>> the initial database so I won't be able to take that one out.
>>>>
>>>> Juhan
>>>>
>>>>
>>>> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval R, 29.
>>>> märts 2019 kell 04:17:
>>>>
>>>> > Hey Juhan,
>>>> >
>>>> > I have updated the Pull request to fineract-cn-provisioner with a
>>>> commit
>>>> > [1] which fixes the errors you're experiencing.
>>>> >
>>>> > I'll advise that you test using option -Ddemoserver.persistent=true
>>>> and
>>>> > -Ddemoserver.lite=true and we're testing just the minimal set of
>>>> services
>>>> > now. In fact,I tested and the tail of my logs showed thus
>>>> > *identity-v1 address:http://localhost:2021/identity/v1
>>>> > <
>>>> >
>>>> https://www.google.com/url?q=http://localhost:2021/identity/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNEjCID96mw3LF1ZIPmjEIQkHh_LcQ
>>>> > >*
>>>> >
>>>> > *office-v1 address:http://localhost:2023/office/v1
>>>> > <
>>>> >
>>>> https://www.google.com/url?q=http://localhost:2023/office/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNGjyk1BHroLddOTd0FhJUJG4zdVag
>>>> > >customer-v1
>>>> > address:http://localhost:2024/customer/v1
>>>> > <
>>>> >
>>>> https://www.google.com/url?q=http://localhost:2024/customer/v1&sa=D&source=hangouts&ust=1553910362599000&usg=AFQjCNGfcoJZIxmcsX34YeEXRUzS0oC2hA
>>>> > >*
>>>> > Albeit, this error [2] turned up related to the Notifications
>>>> service. Just
>>>> > ignore it for now as it will be handled down the line during extensive
>>>> > integration testing.
>>>> >
>>>> > We can keep testing using -Ddemoserver.provision=true for later too
>>>> when
>>>> > we've migrated and tested the individual services.
>>>> >
>>>> > Can you kindly update your provisioner repository and test once more
>>>> and
>>>> > let's keep fixing ?
>>>> >
>>>> > [1]
>>>> >
>>>> >
>>>> https://github.com/Izakey/fineract-cn-provisioner/commit/b83331b723f175003ccf069764414e4ab371662c
>>>> >
>>>> > [2] https://pastebin.com/DBzNyHir
>>>> >
>>>> > Cheers,
>>>> > Isaac Kamga.
>>>> >
>>>> > On Wed, Mar 27, 2019 at 10:23 PM Isaac Kamga <is...@mifos.org>
>>>> > wrote:
>>>> >
>>>> > > Hello Juhan,
>>>> > >
>>>> > > Thank you for taking the time to do some testing.
>>>> > >
>>>> > > On Wed, Mar 27, 2019 at 10:56 AM Juhan Aasaru <aa...@gmail.com>
>>>> wrote:
>>>> > >
>>>> > >> Hi Isaac,
>>>> > >>
>>>> > >> great initiative towards pushing to PostgreSQL.
>>>> > >>
>>>> > >> I looked at your instructions and tried to run your changes
>>>> locally.
>>>> > >> The sh script came in handy to quickly clone and build everything
>>>> that
>>>> > was
>>>> > >> needed.
>>>> > >> But then I got confused about provisioning.
>>>> > >>
>>>> > >> Question #1: How exactly I should run demo-server? I tried the
>>>> following
>>>> > >> options:
>>>> > >>
>>>> > >> 1. I didn't have postgre and cassandra running locally.
>>>> > >> java -jar -Ddemoserver.lite=true
>>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>> > >>
>>>> > >> resulted
>>>> > >>
>>>> > >> org.springframework.beans.factory.BeanCreationException: Error
>>>> creating
>>>> > >> bean with name 'provisionerInitializer':
>>>> > >> Invocation of init method failed; nested exception is
>>>> > >> java.lang.IllegalStateException: Could not initialize service!
>>>> > >> Caused by: org.postgresql.util.PSQLException: FATAL: database
>>>> "seshat"
>>>> > >> does
>>>> > >> not exist
>>>> > >>
>>>> > >
>>>> > > I haven't done testing using this option yet.
>>>> > >
>>>> > >
>>>> > >>
>>>> > >> 2. I started posgresql (and created a database seshat) and
>>>> cassandra and
>>>> > >> ran:
>>>> > >> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>>>> > >> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 \
>>>> > >> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>>> > >> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>> > >>
>>>> > >> Initially applications started up one after anther
>>>> > >> (ProvisionerApplication,
>>>> > >> RhythmApplication, OfficeApplication)
>>>> > >> but CustomerApplication threw an exception:
>>>> > >>
>>>> > >> 11:50:39.377 [qtp1424482154-21] WARN
>>>> o.e.jetty.servlet.ServletHandler -
>>>> > >> org.springframework.web.util.NestedServletException: Request
>>>> processing
>>>> > >> failed; nested exception is java.lang.IllegalStateException: Could
>>>> not
>>>> > >> insert org.apache.fineract.cn.provisioner.tenant info!
>>>> > >> Caused by: org.postgresql.util.PSQLException: ERROR: relation
>>>> > >> "seshat.tenants" does not exist
>>>> > >>
>>>> > >
>>>> > > Try running the same command above without the provision=true
>>>> option and
>>>> > > share what you get.
>>>> > >
>>>> > > I had to create database seshat in postgresql and the can't find
>>>> database
>>>> > > error went away. Down the road, also encountered the same error
>>>> > > <https://pastebin.com/qCT1T5KM> "Could not insert ....
>>>> > provisioner.tenant
>>>> > > info !"
>>>> > > I'm currently working on resolving this.
>>>> > >
>>>> > >
>>>> > >>
>>>> > >> Question #2 - do I still need to have Mysql running? I see that you
>>>> > >> haven't
>>>> > >> made code changes to identity
>>>> > >> which as I understand still uses Mysql.
>>>> > >>
>>>> > >
>>>> > > I don't think you need MySQL running to test the migrations
>>>> currently
>>>> > > underway and there was no need for code changes in the identity
>>>> service.
>>>> > > However, it could be helpful to see the structure of the databases
>>>> in
>>>> > MySQL
>>>> > > so that it helps with debugging and testing.
>>>> > >
>>>> > > Whenever you're available, tell me so we schedule a call so I look
>>>> into
>>>> > > what your experiencing too.
>>>> > >
>>>> > > Cheers,
>>>> > > Isaac Kamga.
>>>> > >
>>>> > >
>>>> > >>
>>>> > >> Kind regards
>>>> > >> Juhan
>>>> > >>
>>>> > >>
>>>> > >> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval
>>>> L, 23.
>>>> > >> märts 2019 kell 04:31:
>>>> > >>
>>>> > >> > Hello fineracters,
>>>> > >> >
>>>> > >> > Trust that you had a great week.
>>>> > >> >
>>>> > >> > I just finished migrating libraries PostgreSQL[1] and Test[2]
>>>> from
>>>> > >> MariaDB
>>>> > >> > to PostgreSQL, mostly adapting SQL statements & database
>>>> connections
>>>> > to
>>>> > >> > work with PostgreSQL and reverting back to Hibernate ORM until
>>>> > database
>>>> > >> > migrations are completed.
>>>> > >> >
>>>> > >> > I have also migrated the "minimal" services viz provisioner[3],
>>>> > >> rhythm[4],
>>>> > >> > office[5], template[6] and customer[7] from MariaDB to
>>>> PostgreSQL. The
>>>> > >> > Spring datasource credentials had to be incorporated into
>>>> > >> application.yaml
>>>> > >> > files to make things work too.
>>>> > >> >
>>>> > >> > Given I updated service-starter [8] and demo-server [9],
>>>> developers
>>>> > can
>>>> > >> > test the migrated services when running the
>>>> *-Ddemoserver.lite=true*
>>>> > >> option
>>>> > >> > which builds the minimal set of services.
>>>> > >> >
>>>> > >> > You'll observe that some of the pull requests are heading to
>>>> develop
>>>> > >> > branches ( It appears comparing changes and creating PRs towards
>>>> the
>>>> > >> same
>>>> > >> > branch name from origin to upstream expires after a few minutes
>>>> on
>>>> > >> > Github... weird )
>>>> > >> >
>>>> > >> > Here is a script [10] to enable anyone clone and do an
>>>> > integration-test
>>>> > >> of
>>>> > >> > the new changes using demo-server.
>>>> > >> >
>>>> > >> > Kindly test these changes and provide feedback.
>>>> > >> >
>>>> > >> > At Your Service,
>>>> > >> > Isaac Kamga.
>>>> > >> >
>>>> > >> > [1] https://github.com/apache/fineract-cn-postgresql/pull/1
>>>> > >> > [2] https://github.com/apache/fineract-cn-test/pull/7
>>>> > >> > [3] https://github.com/apache/fineract-cn-provisioner/pull/6
>>>> > >> > [4] https://github.com/apache/fineract-cn-rhythm/pull/8
>>>> > >> > [5] https://github.com/apache/fineract-cn-office/pull/9
>>>> > >> > [6] https://github.com/apache/fineract-cn-template/pull/6
>>>> > >> > [7] https://github.com/apache/fineract-cn-customer/pull/10
>>>> > >> > [8] https://github.com/apache/fineract-cn-service-starter/pull/8
>>>> > >> > [9] https://github.com/apache/fineract-cn-demo-server/pull/27
>>>> > >> > [10]
>>>> > >> >
>>>> > >> >
>>>> > >>
>>>> >
>>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>>>> > >> >
>>>> > >>
>>>> > >
>>>> >
>>>>
>>>
Re: updates On Migration from MariaDB to PostgreSQL
Posted by Ebenezer Graham <eg...@alustudent.com>.
Hello Isaac/Juhan,
I have tried testing the code in lite mode for 3 times and I got the error
below.
https://pastebin.com/TaJqSNjQ
Before the failure, I got this info.
16:56:03.519 [qtp1262607708-22] INFO o.a.f.c.l.c.ServiceExceptionFilter -
Responding with a service error ServiceError{code=409, message='The given
identity instance didnt recognize the system token as valid. Perhaps the
system keys for the provisioner or for the identity manager are
misconfigured?'}
16:57:03.620 [qtp1262607708-20] INFO o.a.f.c.l.c.ServiceExceptionFilter -
Responding with a service error ServiceError{code=409, message='The given
identity instance didnt recognize the system token as valid. Perhaps the
system keys for the provisioner or for the identity manager are
misconfigured?'}
I will keep testing to get more insights into the error.
*At your service,*
*Ebenezer Graham*
*BSc (Hons) Computing*
[image: EmailSignature.png]
African Leadership University,
Power Mill Road, Pamplemousses,
Mauritius.
GitHub <https://github.com/ebenezergraham> | LinkedIn
<https://www.linkedin.com/in/ebenezer-graham/> | ebenezergraham.me
*Skype*:
ebenezer.graham <http://ebenezergraham.me/>
*This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the sender and then
delete this email. If you are not the intended recipient you are notified
that disclosing, copying, distributing or taking any action in reliance on
the contents of this information is strictly prohibited.*
On Thu, 2 May 2019 at 06:38, Isaac Kamga <is...@mifos.org> wrote:
> Hello Juhan,
>
> I've updated the PRs with commits to fix the errors you were experiencing.
>
> Could you kindly wipe Cassandra and PostgreSQL clean, test once more and
> provide feedback on how it goes ?
>
> Cheers,
> Isaac Kamga.
>
> On Tue, Apr 9, 2019 at 4:36 PM Isaac Kamga <is...@mifos.org> wrote:
>
>> Hey Juhan,
>>
>> Thanks for testing.
>>
>> Wipe clean your Cassandra keyspaces first.
>>
>> I suspect that what's interferring is the keyspaces from your last run
>> when using MariaDB.
>>
>> Please provide feedback how it goes.
>>
>> Cheers,
>> Isaac Kamga.
>>
>> On Sun, Mar 31, 2019 at 10:00 AM Juhan Aasaru <aa...@gmail.com> wrote:
>>
>>> I Isaac,
>>>
>>> I pulled the changes and rebuilt the projects.
>>>
>>> Now when I run (postgres and cassandra working locally):
>>>
>>> java -Ddemoserver.persistent=true -Ddemoserver.provision=true
>>> -Dcustom.cassandra.contactPoints=127.0.0.1:9042
>>> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>>
>>> I get an error here:
>>>
>>> 11:18:42.066 [main] INFO o.s.b.c.e.j.JettyEmbeddedServletContainer -
>>> Jetty
>>> started on port(s) 2024 (http/1.1)
>>> 11:18:42.067 [main] INFO o.s.c.n.e.EurekaDiscoveryClientConfiguration -
>>> Updating port to 2024
>>> 11:18:42.072 [main] INFO o.a.f.c.customer.CustomerApplication - Started
>>> CustomerApplication in 16.423 seconds (JVM running for 17.342)
>>> 11:18:57.435 [qtp1479696465-21] INFO o.a.f.c.l.c.ServiceExceptionFilter
>>> -
>>> Responding with a service error ServiceError{code=409,
>>> message='Application
>>> identity-v1 already exists!'}
>>> 11:18:57.464 [Thread-16] INFO
>>> o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing
>>>
>>> org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@4d50efb8
>>> :
>>> startup date [Sun Mar 31 11:12:52 EEST 2019]; parent:
>>>
>>> org.springframework.context.annotation.AnnotationConfigApplicationContext@75a1cd57
>>>
>>> I understand I have to use -Ddemoserver.provision=true flag to provision
>>> the initial database so I won't be able to take that one out.
>>>
>>> Juhan
>>>
>>>
>>> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval R, 29.
>>> märts 2019 kell 04:17:
>>>
>>> > Hey Juhan,
>>> >
>>> > I have updated the Pull request to fineract-cn-provisioner with a
>>> commit
>>> > [1] which fixes the errors you're experiencing.
>>> >
>>> > I'll advise that you test using option -Ddemoserver.persistent=true and
>>> > -Ddemoserver.lite=true and we're testing just the minimal set of
>>> services
>>> > now. In fact,I tested and the tail of my logs showed thus
>>> > *identity-v1 address:http://localhost:2021/identity/v1
>>> > <
>>> >
>>> https://www.google.com/url?q=http://localhost:2021/identity/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNEjCID96mw3LF1ZIPmjEIQkHh_LcQ
>>> > >*
>>> >
>>> > *office-v1 address:http://localhost:2023/office/v1
>>> > <
>>> >
>>> https://www.google.com/url?q=http://localhost:2023/office/v1&sa=D&source=hangouts&ust=1553910362598000&usg=AFQjCNGjyk1BHroLddOTd0FhJUJG4zdVag
>>> > >customer-v1
>>> > address:http://localhost:2024/customer/v1
>>> > <
>>> >
>>> https://www.google.com/url?q=http://localhost:2024/customer/v1&sa=D&source=hangouts&ust=1553910362599000&usg=AFQjCNGfcoJZIxmcsX34YeEXRUzS0oC2hA
>>> > >*
>>> > Albeit, this error [2] turned up related to the Notifications service.
>>> Just
>>> > ignore it for now as it will be handled down the line during extensive
>>> > integration testing.
>>> >
>>> > We can keep testing using -Ddemoserver.provision=true for later too
>>> when
>>> > we've migrated and tested the individual services.
>>> >
>>> > Can you kindly update your provisioner repository and test once more
>>> and
>>> > let's keep fixing ?
>>> >
>>> > [1]
>>> >
>>> >
>>> https://github.com/Izakey/fineract-cn-provisioner/commit/b83331b723f175003ccf069764414e4ab371662c
>>> >
>>> > [2] https://pastebin.com/DBzNyHir
>>> >
>>> > Cheers,
>>> > Isaac Kamga.
>>> >
>>> > On Wed, Mar 27, 2019 at 10:23 PM Isaac Kamga <is...@mifos.org>
>>> > wrote:
>>> >
>>> > > Hello Juhan,
>>> > >
>>> > > Thank you for taking the time to do some testing.
>>> > >
>>> > > On Wed, Mar 27, 2019 at 10:56 AM Juhan Aasaru <aa...@gmail.com>
>>> wrote:
>>> > >
>>> > >> Hi Isaac,
>>> > >>
>>> > >> great initiative towards pushing to PostgreSQL.
>>> > >>
>>> > >> I looked at your instructions and tried to run your changes locally.
>>> > >> The sh script came in handy to quickly clone and build everything
>>> that
>>> > was
>>> > >> needed.
>>> > >> But then I got confused about provisioning.
>>> > >>
>>> > >> Question #1: How exactly I should run demo-server? I tried the
>>> following
>>> > >> options:
>>> > >>
>>> > >> 1. I didn't have postgre and cassandra running locally.
>>> > >> java -jar -Ddemoserver.lite=true
>>> demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>> > >>
>>> > >> resulted
>>> > >>
>>> > >> org.springframework.beans.factory.BeanCreationException: Error
>>> creating
>>> > >> bean with name 'provisionerInitializer':
>>> > >> Invocation of init method failed; nested exception is
>>> > >> java.lang.IllegalStateException: Could not initialize service!
>>> > >> Caused by: org.postgresql.util.PSQLException: FATAL: database
>>> "seshat"
>>> > >> does
>>> > >> not exist
>>> > >>
>>> > >
>>> > > I haven't done testing using this option yet.
>>> > >
>>> > >
>>> > >>
>>> > >> 2. I started posgresql (and created a database seshat) and
>>> cassandra and
>>> > >> ran:
>>> > >> java -Ddemoserver.provision=true -Ddemoserver.persistent=true
>>> > >> -Dcustom.cassandra.contactPoints=127.0.0.1:9042 \
>>> > >> -Dcassandra.cluster.user=cassandra -Dcassandra.cluster.pwd=password
>>> > >> -Ddemoserver.lite=true -jar demo-server-0.1.0-BUILD-SNAPSHOT.jar
>>> > >>
>>> > >> Initially applications started up one after anther
>>> > >> (ProvisionerApplication,
>>> > >> RhythmApplication, OfficeApplication)
>>> > >> but CustomerApplication threw an exception:
>>> > >>
>>> > >> 11:50:39.377 [qtp1424482154-21] WARN
>>> o.e.jetty.servlet.ServletHandler -
>>> > >> org.springframework.web.util.NestedServletException: Request
>>> processing
>>> > >> failed; nested exception is java.lang.IllegalStateException: Could
>>> not
>>> > >> insert org.apache.fineract.cn.provisioner.tenant info!
>>> > >> Caused by: org.postgresql.util.PSQLException: ERROR: relation
>>> > >> "seshat.tenants" does not exist
>>> > >>
>>> > >
>>> > > Try running the same command above without the provision=true option
>>> and
>>> > > share what you get.
>>> > >
>>> > > I had to create database seshat in postgresql and the can't find
>>> database
>>> > > error went away. Down the road, also encountered the same error
>>> > > <https://pastebin.com/qCT1T5KM> "Could not insert ....
>>> > provisioner.tenant
>>> > > info !"
>>> > > I'm currently working on resolving this.
>>> > >
>>> > >
>>> > >>
>>> > >> Question #2 - do I still need to have Mysql running? I see that you
>>> > >> haven't
>>> > >> made code changes to identity
>>> > >> which as I understand still uses Mysql.
>>> > >>
>>> > >
>>> > > I don't think you need MySQL running to test the migrations currently
>>> > > underway and there was no need for code changes in the identity
>>> service.
>>> > > However, it could be helpful to see the structure of the databases in
>>> > MySQL
>>> > > so that it helps with debugging and testing.
>>> > >
>>> > > Whenever you're available, tell me so we schedule a call so I look
>>> into
>>> > > what your experiencing too.
>>> > >
>>> > > Cheers,
>>> > > Isaac Kamga.
>>> > >
>>> > >
>>> > >>
>>> > >> Kind regards
>>> > >> Juhan
>>> > >>
>>> > >>
>>> > >> Kontakt Isaac Kamga (<is...@mifos.org>) kirjutas kuupäeval
>>> L, 23.
>>> > >> märts 2019 kell 04:31:
>>> > >>
>>> > >> > Hello fineracters,
>>> > >> >
>>> > >> > Trust that you had a great week.
>>> > >> >
>>> > >> > I just finished migrating libraries PostgreSQL[1] and Test[2] from
>>> > >> MariaDB
>>> > >> > to PostgreSQL, mostly adapting SQL statements & database
>>> connections
>>> > to
>>> > >> > work with PostgreSQL and reverting back to Hibernate ORM until
>>> > database
>>> > >> > migrations are completed.
>>> > >> >
>>> > >> > I have also migrated the "minimal" services viz provisioner[3],
>>> > >> rhythm[4],
>>> > >> > office[5], template[6] and customer[7] from MariaDB to
>>> PostgreSQL. The
>>> > >> > Spring datasource credentials had to be incorporated into
>>> > >> application.yaml
>>> > >> > files to make things work too.
>>> > >> >
>>> > >> > Given I updated service-starter [8] and demo-server [9],
>>> developers
>>> > can
>>> > >> > test the migrated services when running the
>>> *-Ddemoserver.lite=true*
>>> > >> option
>>> > >> > which builds the minimal set of services.
>>> > >> >
>>> > >> > You'll observe that some of the pull requests are heading to
>>> develop
>>> > >> > branches ( It appears comparing changes and creating PRs towards
>>> the
>>> > >> same
>>> > >> > branch name from origin to upstream expires after a few minutes on
>>> > >> > Github... weird )
>>> > >> >
>>> > >> > Here is a script [10] to enable anyone clone and do an
>>> > integration-test
>>> > >> of
>>> > >> > the new changes using demo-server.
>>> > >> >
>>> > >> > Kindly test these changes and provide feedback.
>>> > >> >
>>> > >> > At Your Service,
>>> > >> > Isaac Kamga.
>>> > >> >
>>> > >> > [1] https://github.com/apache/fineract-cn-postgresql/pull/1
>>> > >> > [2] https://github.com/apache/fineract-cn-test/pull/7
>>> > >> > [3] https://github.com/apache/fineract-cn-provisioner/pull/6
>>> > >> > [4] https://github.com/apache/fineract-cn-rhythm/pull/8
>>> > >> > [5] https://github.com/apache/fineract-cn-office/pull/9
>>> > >> > [6] https://github.com/apache/fineract-cn-template/pull/6
>>> > >> > [7] https://github.com/apache/fineract-cn-customer/pull/10
>>> > >> > [8] https://github.com/apache/fineract-cn-service-starter/pull/8
>>> > >> > [9] https://github.com/apache/fineract-cn-demo-server/pull/27
>>> > >> > [10]
>>> > >> >
>>> > >> >
>>> > >>
>>> >
>>> https://github.com/Izakey/fineract-cn-demo-server/blob/FINCN-141/scripts/Unix/postgres-setup-lite.sh
>>> > >> >
>>> > >>
>>> > >
>>> >
>>>
>>