Re: Announcing a Scalability and Performance benchmark test

[Ankit Muellner <an...@muellners.org>]

@Ed Cable <ed...@mifos.org>  I have got updated with the
community's ongoing efforts on this thread. Joined discourse working group
too.
The direction is right in terms of setting up replicable tools for multiple
use cases to be tested  for performance and scalability optimization. Also
agree with other suggestions that each periodic Test can be tied up with
the PR mechanism to fill the GAPs.
Will move this dialogue there and closing this thread.

Thanks everybody for your inputs.


On Sun, Jul 19, 2020 at 3:41 AM Percy Enoabane <pe...@gmail.com>
wrote:

> Hi all, I would like to contribute to this front. Can anyone help me in
> defining suitable scenarios which would allow me to define some postman
> scripts which could be used for scale testing.
>
> On Wed, Jul 15, 2020 at 11:46 PM Michael Vorburger <mi...@vorburger.ch>
> wrote:
>
>> This is an old and recurring topic... and one that in the past ultimately
>> never really substantially progressed. What the Fineract community actually
>> really needs most re. performance, in order of priorities, IMHO is:
>>
>> 1. First and foremost, most importantly, someone stepping up making an
>> initial contribution that makes it easy enough for any community
>> contributor to run Load Testing. Think your JMeter, Postman, even custom
>> Java code (seriously, why not, may be simplest?) that appropriately
>> simulates real world workload. We don't really have anything like that
>> (that I know of). Anyone contributing anything along those lines would
>> unlock a multiplier effect enabling others to subsequently contribute DB
>> query improvements, do serious Java profiling, have recurring performance
>> evaluations e.g. per release, etc. etc.
>>
>> 2. Contributions of Pull Requests fixing specific performance issues
>> found. Whether adding missing SQL indices, JPA over eager pre-fetch, or
>> hard core  optimizing "hot code paths" in Java - don't write about it -
>> send PRs!
>>
>> Sharing test results of what scale someone has been able to achieve with
>> their own testing using scripts for tools that are not publicly shared,
>> thus not repeatable by others in the community, preferably after making
>> "downstream" (non shared back) improvements, are of very limited community
>> value, IMHO.
>>
>>
>> On Tue, 14 Jul 2020, 22:39 Ed Cable, <ed...@mifos.org> wrote:
>>
>>> Hi Saransh,
>>>
>>> Thank you for raising the topic of Performance and Scalability to the
>>> level of the mailing list. This is squarely in line with the ongoing
>>> efforts of the Performance and Scalability working group that was kicked
>>> off in October of 2019 - https://markmail.org/thread/ebgdyqpjk3l4e7d7 and
>>> can be found on Discourse at https://discourse.mifos.org/g/performance
>>>
>>> This wiki page -
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=133632146 was
>>> tracking the efforts of the group and how individuals from the community
>>> could get involved. It also includes links to the previous performance
>>> benchmarking efforts that the community had led including the IBM one  -
>>> https://www.ibm.com/partnerworld/page/stg_ast_sys-mifos-x-on-ibm-powerlinux-servers
>>>
>>> Sam, thanks for your feedback. It's exactly aligned with an action item
>>> for how we wanted to get the community involved with making available
>>> replicable tools to help with ongoing performance testing. We welcome any
>>> volunteers or partners that would like to assist to step in and participate.
>>>
>>> Victor, I can work with you to best document and share what you've done
>>> around Fineract 1.x and Fineract CN as we've been eagerly anticipating
>>> sharing your work with the community.
>>>
>>> Thanks,
>>>
>>> Ed
>>>
>>> On Tue, Jul 14, 2020 at 12:57 PM Tonio O <tc...@gmail.com> wrote:
>>>
>>>> Hi Ankit,
>>>> What is the hardware specification and deployment architecture used in
>>>> these benchmarks?
>>>>
>>>> Regards,
>>>> Tcbuzor
>>>>
>>>> On Tue, Jul 14, 2020 at 2:24 AM Ankit Muellner <an...@muellners.org>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> In wide variety of Performance and Scalability Studies conducted over
>>>>> last 5 years, following are the aspects reported:
>>>>>
>>>>> In a 2015 study, it has been found that:
>>>>>
>>>>>
>>>>>    1.
>>>>>
>>>>>    Platform can support over 3000 virtual users on a single server
>>>>>    with mean login times under 2000ms.
>>>>>    2.
>>>>>
>>>>>    Platform's client management easily supports 3000 users while
>>>>>    accessing client profile from 4,910,000 clients in the clients management
>>>>>    system
>>>>>    3.
>>>>>
>>>>>    Platform's client management easily supports 3000 users while
>>>>>    accessing loan profiles from 20M loans in the LMS module
>>>>>    4.
>>>>>
>>>>>    Platform can handle 2800 concurrent repayments with mean response
>>>>>    time of 3500ms.
>>>>>    5.
>>>>>
>>>>>    Given sufficient database resources and efficient load balancing,
>>>>>    platform can scale linearly as one adds additional servers to a cluster.
>>>>>
>>>>> Citation: Conflux Mifos Partner
>>>>>
>>>>> In a 2017 study, it has been found that:
>>>>>
>>>>>    1.
>>>>>
>>>>>    The platform when run by 1500 users executed 8.8
>>>>>    throughput(requests processed per second by the test server) with a Std.
>>>>>    deviation of 14538.84 and an error margin of 39.73% for one user.
>>>>>    2.
>>>>>
>>>>>    Further, the study suggests that 1332.2 bytes of average date of
>>>>>    the application is a generic output with 11.52 KB/sec of application data.
>>>>>    3.
>>>>>
>>>>>    It is also reported that of the 1500 HTTP requests, 904 are OK,
>>>>>    570 timed out and 19 were Reset by the platform. Remaining are reported as
>>>>>    multiple type of errors such as 443: failed to respond.
>>>>>
>>>>>    Citation: Esolve
>>>>>
>>>>>
>>>>>    *There have not been any new study reported to the community. Of
>>>>>    course there are Hosting experiences from partners.*
>>>>>
>>>>>    *We are planning a newer 2020 benchmark study with our case study
>>>>>    partners- both are leading FIs in their industry by Sept this year. We are
>>>>>    planning to use IBM cloud infrastructure and including containerization
>>>>>    aspects in this study. We will be releasing it to support community
>>>>>    documentation. This will cover CN & 1.x.*
>>>>>
>>>>>    *Anyone would like to participate or comment over this thread on
>>>>>    the factors to consider? Is there a study which a partner has performed but
>>>>>    not reported?*
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Ankit Müllner
>>>>> Muellners Europe
>>>>> +4581929792
>>>>>
>>>>> Sent from my iPad.
>>>>>
>>>>> This mail is governed by Muellners® IT policy.
>>>>> The information contained in this e-mail and any accompanying
>>>>> documents may contain information that is confidential or otherwise
>>>>> protected from disclosure. If you are not the intended recipient of this
>>>>> message, or if this message has been addressed to you in error, please
>>>>> immediately alert the sender by reply e-mail and then delete this message,
>>>>> including any attachments. Any dissemination, distribution or other use of
>>>>> the contents of this message by anyone other than the intended recipient is
>>>>> strictly prohibited. All messages sent to and from this e-mail address may
>>>>> be monitored as permitted by applicable law and regulations to ensure
>>>>> compliance with our internal policies and to protect our business. E-mails
>>>>> are not secure and cannot be guaranteed to be error free as they can be
>>>>> intercepted, amended, lost or destroyed, or contain viruses. You are deemed
>>>>> to have accepted these risks if you communicate with us by e-mail.
>>>>>
>>>>
>>>
>>> --
>>> *Ed Cable*
>>> President/CEO, Mifos Initiative
>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>>
>>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
>>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>>>
>>>

-- 
Ankit Müllner
Muellners LLC

This mail is governed by Muellners®  IT policy.
The information contained in this e-mail and any accompanying documents may
contain information that is confidential or otherwise protected from
disclosure. If you are not the intended recipient of this message, or if
this message has been addressed to you in error, please immediately alert
the sender by reply e-mail and then delete this message, including any
attachments. Any dissemination, distribution or other use of the contents
of this message by anyone other than the intended recipient is strictly
prohibited. All messages sent to and from this e-mail address may be
monitored as permitted by applicable law and regulations to ensure
compliance with our internal policies and to protect our business. E-mails
are not secure and cannot be guaranteed to be error free as they can be
intercepted, amended, lost or destroyed, or contain viruses. You are deemed
to have accepted these risks if you communicate with us by e-mail.