Re: Scalability & Performance Working Group

[Ed Cable <ed...@mifos.org>]

Hi everyone,

The response and participation on this thread was great to see. Just like
we did with the Open Banking Working Group, I've set up a Discourse Group
for Performance and Scalability. With all the interest in being involved,
I'd like for people to just start stepping up and participating.

The goals are the same as articulated below and I know from conversations
with multiple partners that there are ongoing efforts related to
scalability on both Fineract 1.x and Fineract CN so would encourage those
individuals to try to make those contributions through this working group.

Please visit https://discourse.mifos.org/g/performance/ to join the working
group and we'll have one central place to house the discussion and continue
posting artifacts and resources to the wiki.

One good first item upon joining the group would be to introduce yourself,
the current work you're doing around performance/scalability, and what/how
you'd like to contribute.

With a clearer picture of priorities, we'll then actionably work on
prioritizing these through the roadmap and across releases, sharing current
resources publicly, splitting responsibilities amongst
partners/individuals, assign for projects like GSOC, etc.

Thanks,

Ed

On Wed, Oct 23, 2019 at 12:59 PM Ed Cable <ed...@mifos.org> wrote:

> Hi Mifos and Fineract communities,
>
> In follow up to my previous comments on Joseph's thread I wanted to
> initiate a collaborative community-wide effort to help address the ongoing
> and growing need for helping Mifos/Fineract scale and sustain high load
> environments.
>
> This really represent an area where we can demonstrate that together as
> Open Source Community we are much stronger than each individual or partner
> trying to tackle this alone.
>
> This would be for both Fineract and Fineract CN.
>
> *Why the need for this Working Group?*
> The formation of a working group focused on this subject is needed as
> evident from various mail thread or support requests trying to optimize the
> system for high load/volume environments and a growing number of prospects
> eager to use the platform but needing more visibility into its performance
> and ability to meet high TPS requirements.
>
> Secondly, there are many partner-led implementations and deployments of
> Fineract/Mifos supporting millions of clients and it would be valuable to
> share that knowledge across the community.
>
> Lastly, it's been quite some time since we've had public performance
> testing done, the efforts led by eSolve in 2017 (see wiki page) and IBM/Conflux
> in 2015
> <https://www.ibm.com/partnerworld/page/stg_ast_sys-mifos-x-on-ibm-powerlinux-servers>both
> pre-date the recent change from Hibernate to OpenJPA. Kumaranath with
> support from Avik of Fynarfin worked on performance related issues during
> 2018 GSOC -
> https://docs.google.com/document/d/18_awblHsI3uZmc7f80Q5HljAGwc2XhIpYOlOtW_-In0
>
>
> *What/How*
>
>    1. Create a set of reproducible tools to enable automation of this on
>    an ongoing basis.
>       1. Performance testing related data sets - i.e. DB dumps, scripts
>       to generate test data
>       2. Postman scripts to run load tests
>       3. Perhaps we can create a repo on github for this?
>    2. Access to Resources
>       1. If you have hardware or cloud environments available to conduct
>       these load/performance tests please share.
>    3. Culture & Transparency - Establish this as a priority area of the
>    community to collectively address.
>    4. Documentation (likely on the Fineract Wiki) - I created this page
>    to start: https://cwiki.apache.org/confluence/x/khD3Bw
>       1. Share and document existing results of performance testing to
>       date. Share the details of environments you have set up to address high
>       load needs.
>       2. Document the scenarios and TPS requirements that need to be
>       tested
>       3. Document typical improvements to address performance (at a
>       configuration, database, code, level, etc.)
>    5. Identify issues to fix
>       1. Log new issues, update existing issues, and tag appropriately.
>    6. Share code and fixes to address performance issues
>       1. Make these a priority and plan out in our release roadmapping.
>
> *Next Steps*
>
>    - If interested in joining the group, respond to this thread.
>    - Begin sharing your inputs on the wiki page.
>    - Start fixing existing issues.
>    - Help in creating and executing on our performance testing plan.
>
> Thanks,
>
> Ed
>
>

-- 
*Ed Cable*
President/CEO, Mifos Initiative
edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649

*Collectively Creating a World of 3 Billion Maries | *http://mifos.org
<http://facebook.com/mifos>  <http://www.twitter.com/mifos>