Re: Migration to MySQL 5.7

[Ed Cable <ed...@mifos.org>]

Steve,

To follow up on this thread, did you make all the changes you need to make
and are ready for this go through QA? If so, I'd have you open a pull
request for this and we'll merge it into the develop branch for Fineract
1.2 and I'll have Santosh begin on it.

Ed

On Mon, Feb 26, 2018 at 1:03 PM, Steve Conrad <sc...@gmail.com> wrote:

> Hi all -
>
> I have created a new branch on my fork for MySQL 5.7. I have gone through
> all of the migration scripts as well as all of the dynamic SQL in the
> Fineract codebase (all of the services code, etc) to ensure that it is
> compatible with 5.7. We are using strict mode (ONLY_FULL_GROUP_BY enabled).
> You are welcome to work from this tree and can let me know if you have any
> issues running on 5.7:
>
> https://github.com/conradsp/fineract/tree/mysql-5.7
>
> I am not sure if this change will be a part of Fineract 1.2 or 1.3. I would
> propose merging these changes into the develop branch when we start work on
> the appropriate release.
>
> Thanks,
> Steve
>
>
> On Thu, Feb 22, 2018 at 9:36 AM, Sendoro Juma <se...@singo.africa>
> wrote:
>
> > Hello Steve,
> >
> > My input:-
> >
> > Option 2: with plan for Release 1.3 o to ensure a thorough test as you
> put
> > it right.
> >
> > Cheers
> > Sendoro
> >
> > ----- Original Message -----
> > From: "Steve Conrad" <sc...@gmail.com>
> > To: "dev" <de...@fineract.apache.org>
> > Cc: "mifos-developer" <mi...@lists.sourceforge.net>
> > Sent: Thursday, February 22, 2018 5:29:36 PM
> > Subject: Re: Migration to MySQL 5.7
> >
> > Hi all -
> >
> > I wanted to revisit this conversation.I have been working on this
> > migration, and there are 2 possible approaches that we can take.
> >
> > 1) Require the user to turn off ONLY_FULL_GROUP_BY in sql_mode for the
> > MySQL instance (in /etc/mysql/my.cnf for most users). As of MySQL 5.7.5
> > this is turned on by default. The advantage of this approach is that it
> > requires only 1 change to migration files, and reduces the risks (as
> > pointed out by Nikhil and others) of breaking Read Platform services,
> > Pentaho reports, and any other externally configured queries.
> >
> > 2) Use the default settings where ONLY_FULL_GROUP_BY is on. This requires
> > more changes to the migration scripts and would require more testing of
> > external services and reports.
> >
> > The following link gives some good background on ONLY_FULL_GROUP_BY -
> > http://rpbouman.blogspot.nl/2014/09/mysql-575-group-by-
> > respects-functional.html
> >
> > My recommendation would be to leave ONLY_FULL_GROUP_BY on and do the
> > thorough testing that is required. However, I welcome your input and
> > thoughts.
> > Thanks,
> > Steve
> >
> >
> > On Wed, Jan 17, 2018 at 8:00 PM, Nikhil Pawar <ni...@gmail.com>
> wrote:
> >
> > > Hi Steve,
> > >
> > > +1 from my side as well.
> > > I think the areas where the application directly interacts with
> database
> > (
> > > w/o any intermediate layer like OpenJPA) would be more vulnerable.
> > > The places to look out according to me would be:
> > >  1) Migration scripts (which you are already aware of)
> > >  2) Various ReadPlatform services wherein we have written SQLs to fetch
> > > data from database.
> > > 3) Pentaho Reports (not quite sure about it but this module needs to be
> > > thoroughly tested after migration).
> > >
> > > Regards,
> > > Nikhil
> > >
> > >
> > >
> > > On Wed, Jan 17, 2018 at 8:48 AM, Steve Conrad <sc...@gmail.com>
> > wrote:
> > >
> > > > To the Mifos and Fineract dev communities -
> > > >
> > > > My name is Steve Conrad, and I have recently begun volunteering with
> > the
> > > > Mifos/Fineract community and really appreciate the great work that
> you
> > > all
> > > > are doing.
> > > >
> > > > I wanted to propose a new project to migrate the Fineract platform to
> > > MySQL
> > > > 5.7. There are several features in MySQL 5.7 that would benefit the
> > > project
> > > > - performance, embedded JSON objects in the schema, multithreaded
> > > > replication, among others.
> > > >
> > > > When I installed Fineract locally, I used MySQL 5.7. I had to change
> a
> > > few
> > > > of the database migration scripts. I have obviously not gone through
> > all
> > > of
> > > > the use cases, but so far am running with no issues.
> > > >
> > > > Does anyone know of any reasons why we could not move to MySQL 5.7?
> > > > Obviously there would be a large testing task, but it may be a
> > worthwhile
> > > > investment to keep the Fineract platform current.
> > > >
> > > > Thanks,
> > > > Steve
> > > >
> > >
> >
>



-- 
*Ed Cable*
President/CEO, Mifos Initiative
edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649

*Collectively Creating a World of 3 Billion Maries | *http://mifos.org
<http://facebook.com/mifos>  <http://www.twitter.com/mifos>

Re: Migration to MySQL 5.7

[Steve Conrad <sc...@gmail.com>]

Yes, I believe I have made all of the changes necessary to start the QA
process. I have done some cursory testing, and I ran integration tests. I
did see several failures in the integration tests, but as far as I can
tell, they are not related to the MySQL migration.

I have created a PR for this migration:
https://github.com/apache/fineract/pull/448

Please let me know if you run into any issues related to MySQL 5.7 and I
will take a look at them.
Thanks,
Steve


On Tue, Mar 6, 2018 at 1:13 PM, Ed Cable <ed...@mifos.org> wrote:

> Steve,
>
> To follow up on this thread, did you make all the changes you need to make
> and are ready for this go through QA? If so, I'd have you open a pull
> request for this and we'll merge it into the develop branch for Fineract
> 1.2 and I'll have Santosh begin on it.
>
> Ed
>
> On Mon, Feb 26, 2018 at 1:03 PM, Steve Conrad <sc...@gmail.com> wrote:
>
> > Hi all -
> >
> > I have created a new branch on my fork for MySQL 5.7. I have gone through
> > all of the migration scripts as well as all of the dynamic SQL in the
> > Fineract codebase (all of the services code, etc) to ensure that it is
> > compatible with 5.7. We are using strict mode (ONLY_FULL_GROUP_BY
> enabled).
> > You are welcome to work from this tree and can let me know if you have
> any
> > issues running on 5.7:
> >
> > https://github.com/conradsp/fineract/tree/mysql-5.7
> >
> > I am not sure if this change will be a part of Fineract 1.2 or 1.3. I
> would
> > propose merging these changes into the develop branch when we start work
> on
> > the appropriate release.
> >
> > Thanks,
> > Steve
> >
> >
> > On Thu, Feb 22, 2018 at 9:36 AM, Sendoro Juma <se...@singo.africa>
> > wrote:
> >
> > > Hello Steve,
> > >
> > > My input:-
> > >
> > > Option 2: with plan for Release 1.3 o to ensure a thorough test as you
> > put
> > > it right.
> > >
> > > Cheers
> > > Sendoro
> > >
> > > ----- Original Message -----
> > > From: "Steve Conrad" <sc...@gmail.com>
> > > To: "dev" <de...@fineract.apache.org>
> > > Cc: "mifos-developer" <mi...@lists.sourceforge.net>
> > > Sent: Thursday, February 22, 2018 5:29:36 PM
> > > Subject: Re: Migration to MySQL 5.7
> > >
> > > Hi all -
> > >
> > > I wanted to revisit this conversation.I have been working on this
> > > migration, and there are 2 possible approaches that we can take.
> > >
> > > 1) Require the user to turn off ONLY_FULL_GROUP_BY in sql_mode for the
> > > MySQL instance (in /etc/mysql/my.cnf for most users). As of MySQL 5.7.5
> > > this is turned on by default. The advantage of this approach is that it
> > > requires only 1 change to migration files, and reduces the risks (as
> > > pointed out by Nikhil and others) of breaking Read Platform services,
> > > Pentaho reports, and any other externally configured queries.
> > >
> > > 2) Use the default settings where ONLY_FULL_GROUP_BY is on. This
> requires
> > > more changes to the migration scripts and would require more testing of
> > > external services and reports.
> > >
> > > The following link gives some good background on ONLY_FULL_GROUP_BY -
> > > http://rpbouman.blogspot.nl/2014/09/mysql-575-group-by-
> > > respects-functional.html
> > >
> > > My recommendation would be to leave ONLY_FULL_GROUP_BY on and do the
> > > thorough testing that is required. However, I welcome your input and
> > > thoughts.
> > > Thanks,
> > > Steve
> > >
> > >
> > > On Wed, Jan 17, 2018 at 8:00 PM, Nikhil Pawar <ni...@gmail.com>
> > wrote:
> > >
> > > > Hi Steve,
> > > >
> > > > +1 from my side as well.
> > > > I think the areas where the application directly interacts with
> > database
> > > (
> > > > w/o any intermediate layer like OpenJPA) would be more vulnerable.
> > > > The places to look out according to me would be:
> > > >  1) Migration scripts (which you are already aware of)
> > > >  2) Various ReadPlatform services wherein we have written SQLs to
> fetch
> > > > data from database.
> > > > 3) Pentaho Reports (not quite sure about it but this module needs to
> be
> > > > thoroughly tested after migration).
> > > >
> > > > Regards,
> > > > Nikhil
> > > >
> > > >
> > > >
> > > > On Wed, Jan 17, 2018 at 8:48 AM, Steve Conrad <sc...@gmail.com>
> > > wrote:
> > > >
> > > > > To the Mifos and Fineract dev communities -
> > > > >
> > > > > My name is Steve Conrad, and I have recently begun volunteering
> with
> > > the
> > > > > Mifos/Fineract community and really appreciate the great work that
> > you
> > > > all
> > > > > are doing.
> > > > >
> > > > > I wanted to propose a new project to migrate the Fineract platform
> to
> > > > MySQL
> > > > > 5.7. There are several features in MySQL 5.7 that would benefit the
> > > > project
> > > > > - performance, embedded JSON objects in the schema, multithreaded
> > > > > replication, among others.
> > > > >
> > > > > When I installed Fineract locally, I used MySQL 5.7. I had to
> change
> > a
> > > > few
> > > > > of the database migration scripts. I have obviously not gone
> through
> > > all
> > > > of
> > > > > the use cases, but so far am running with no issues.
> > > > >
> > > > > Does anyone know of any reasons why we could not move to MySQL 5.7?
> > > > > Obviously there would be a large testing task, but it may be a
> > > worthwhile
> > > > > investment to keep the Fineract platform current.
> > > > >
> > > > > Thanks,
> > > > > Steve
> > > > >
> > > >
> > >
> >
>
>
>
> --
> *Ed Cable*
> President/CEO, Mifos Initiative
> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>
> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>