You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@fineract.apache.org by Anjil R Chinnapatlolla <an...@in.ibm.com> on 2023/02/28 19:12:35 UTC
Fineract banking use cases for performance/scalability evaluation
Dear community members,
We are evaluating the performance of OCP (Openshift Container Platform) infrastructure with banking workloads on different hardware architectures. With Fineract as one of the banking workloads, we have built the load test suite with APIs covering below use cases (Majorly with Savings and Loan products).
I request the community members and teams who have Fineract deployed in production or being evaluated, to suggest specific use-cases which are more challenging in nature (in latency and scalability perspective) that we should consider adding to below list while evaluating performance of the deployment infrastructure (which in turn covers the performance of Fineract).
Setup Bank Environment
Category
Use case
Create Codes
Create client closure reason code
Create payment type code
Get payment type codes
Ledger Accounting
Create Asset Ledger Account
Create Liability Ledger Account
Create Revenue Ledger Account
Create Expense Ledger Account
List General Ledger Accounts
Financial Activity Accounting
Map Financial Activity Account
Define Savings Products
Create savings product
Define Loan Products
Create personal loan product
Setup Clients
Create client
Setup Savings Account
Submit savings account
Approve savings account
Activate savings account
Setup Loan Accounts
Submit loan application
Approve loan application
Transactions
Category
Use case
Clients
Get all clients
Deposits
Deposit to savings account
Withdrawals
Withdraw from savings account
Balance Enquiry
Balance enquiry with associations
Balance Enquiry
Post Interest
Post Interest on savings account
Disburse Loans
Disburse loan application
Undo disburse loan application
Disburse loan to savings
List All Loans
List loans
Reporting
Report savings account dormancy
Report client listing
Retrieve a report
List all reports and parameters
Thanks & Regards,
Anjil Reddy Chinnapatlolla
Re: Fineract banking use cases for performance/scalability evaluation
Posted by VICTOR MANUEL ROMERO RODRIGUEZ <vi...@fintecheando.mx>.
Hello,
In hour experience we have faced the following challenges when migrating to
Apache Fineract 1.8
- *Bulk import (which is important for migration from other Core Bankings
running on Mainframe)* HTTPS/API is not the best solution, nor the existing
Excel File for importing the data.
One sample and some specs in our Lab Environment, with a small data set:
MySQL 8.0.32 Server barebone on Dell Workstation - Intel(R) Xeon(R) E-2276M
CPU @ 2.80GHz, 128 GB RAM. 6TB SSD NVM2.
mysql> select count(*) from m_loan;
+----------+
| count(*) |
+----------+
| 1356480 |
+----------+
1 row in set (0,95 sec)
mysql> select count(*) from m_client;
+----------+
| count(*) |
+----------+
| 1356480 |
+----------+
1 row in set (0,38 sec)
mysql> select count(*) from m_loan_repayment_schedule;
+----------+
| count(*) |
+----------+
| 48833316 |
+----------+
1 row in set (17,94 sec)
Network: 10Gbit
6 Apache Fineract Instances on Docker Containers on 2 Servers Running each
one on 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz, 32GB, 1TB NVM2
In all the servers we have set this parameter
vm.swappiness=0
In the docker for Apache Fineract:
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
- JAVA_TOOL_OPTIONS="-Xmx4G"
The best number was around 50,000 clients (with address and activated),
loans (approved/disbursed) using the HTTPS API, we have stopped the test,
because sometimes the HTTP got stale because of Memory. We had to use a
custom JMS setup and JDBC batch saving for completing the task in 48 hours
on the same hardware, which will be used as a metric for the Homologation
and Production setup.
As a NOTE:
Just a note BEFORE the UTC Date/DateTime and other improvements done in
Apache Fineract 1.8 we hit issues for DST in our TimeZone
(America/Mexico_City) in DBs like MySQL, MariaDB and TiDB.
HOUR_OF_DAY: 0 -> 1
Seems that some dates were calculated as DST when being read/retrieved by
the JDBC. Example the date '1950-02-12' (and many others) causes that
exception in Apache Fineract version 1.5, 1.6, 1,7 if the Operating System
of the DB is running in the TZ America/Mexico_City . We notice that the
client was created properly but the savings accounts, transactions and
payments were not properly processed. Solution was to set the UTC on the DB
server and make sure to use UTC in all the connections removing any TZ on
the environment variables. As I have mentioned this is in bulk imports.
Regards
Victor Romero
El dom, 5 mar 2023 a las 23:24, Anjil R Chinnapatlolla (<an...@in.ibm.com>)
escribió:
> Thanks Ed for circulating it across the implementers. Will look forward
> for suggestions from the implementer community to add more use cases to the
> list.
>
>
>
> We are just done with functional evaluation with minimal clients and
> accounts with below mentioned use cases and moving to a more scaled
> environment with increased clients, savings and loan accounts.
>
>
>
> Will keep sharing the observations as we progress with the evaluations.
>
>
>
> There were few issues observed, like Read Pods (Fineract Pod deployed in
> Read only mode) hitting Out Of Memory (due to thread limit exceeding and
> not actual heap) while querying for savings accounts at bulk. Will look
> further into the data and summarize and share observations as we start to
> evaluate with a scaled environment.
>
> Thanks & Regards,
>
> Anjil Reddy Chinnapatlolla
>
> *From: *Ed Cable <ed...@mifos.org>
> *Date: *Thursday, 2 March 2023 at 11:08 AM
> *To: *dev@fineract.apache.org <de...@fineract.apache.org>
> *Subject: *[EXTERNAL] Re: Fineract banking use cases for
> performance/scalability evaluation
>
> Thank you Anjil for the contributions you and the rest of the team are
> making around performance/scalability evaluation. It will be nice to have
> some recent evaluations of performance beyond just for lending/credit. I
> will circulate your request
>
> ZjQcmQRYFpfptBannerStart
>
> *This Message Is From an External Sender *
>
> This message came from outside your organization.
>
> ZjQcmQRYFpfptBannerEnd
>
> Thank you Anjil for the contributions you and the rest of the team are
> making around performance/scalability evaluation. It will be nice to have
> some recent evaluations of performance beyond just for lending/credit. I
> will circulate your request from other implementers across the community.
>
>
>
> Thanks,
>
>
>
> Ed
>
>
>
> On Tue, Feb 28, 2023 at 11:12 AM Anjil R Chinnapatlolla <
> anchinna@in.ibm.com> wrote:
>
> Dear community members,
>
>
>
> We are evaluating the performance of OCP (Openshift Container Platform)
> infrastructure with banking workloads on different hardware architectures.
> With Fineract as one of the banking workloads, we have built the load test
> suite with APIs covering below use cases (Majorly with Savings and Loan
> products).
>
>
>
> I request the community members and teams who have Fineract deployed in
> production or being evaluated, to suggest specific use-cases which are more
> challenging in nature (in latency and scalability perspective) that we
> should consider adding to below list while evaluating performance of the
> deployment infrastructure (which in turn covers the performance of
> Fineract).
>
>
>
>
>
> Setup Bank Environment
>
> Category
>
> Use case
>
> *Create Codes*
>
>
>
>
>
> Create client closure reason code
>
>
>
> Create payment type code
>
>
>
> Get payment type codes
>
> *Ledger Accounting*
>
>
>
>
>
> Create Asset Ledger Account
>
>
>
> Create Liability Ledger Account
>
>
>
> Create Revenue Ledger Account
>
>
>
> Create Expense Ledger Account
>
>
>
> List General Ledger Accounts
>
> *Financial Activity Accounting*
>
>
>
>
>
> Map Financial Activity Account
>
> *Define Savings Products*
>
>
>
>
>
> Create savings product
>
> *Define Loan Products*
>
>
>
>
>
> Create personal loan product
>
> *Setup Clients*
>
>
>
>
>
> Create client
>
> *Setup Savings Account*
>
>
>
>
>
> Submit savings account
>
>
>
> Approve savings account
>
>
>
> Activate savings account
>
> *Setup Loan Accounts*
>
>
>
>
>
> Submit loan application
>
>
>
> Approve loan application
>
>
>
>
>
> *Transactions*
>
> *Category*
>
> *Use case*
>
> *Clients*
>
>
>
>
>
> Get all clients
>
> *Deposits*
>
>
>
>
>
> Deposit to savings account
>
> *Withdrawals*
>
>
>
>
>
> Withdraw from savings account
>
> *Balance Enquiry*
>
>
>
>
>
> Balance enquiry with associations
>
>
>
> Balance Enquiry
>
> *Post Interest *
>
>
>
>
>
> Post Interest on savings account
>
> *Disburse Loans*
>
>
>
>
>
> Disburse loan application
>
>
>
> Undo disburse loan application
>
>
>
> Disburse loan to savings
>
> *List All Loans*
>
>
>
>
>
> List loans
>
> *Reporting *
>
>
>
>
>
> Report savings account dormancy
>
>
>
> Report client listing
>
>
>
> Retrieve a report
>
>
>
> List all reports and parameters
>
>
>
>
>
> Thanks & Regards,
>
> Anjil Reddy Chinnapatlolla
>
>
>
>
> --
>
> *Ed Cable*
>
> President/CEO, Mifos Initiative
>
> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>
>
>
> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
> <http://facebook.com/mifos> <http://www.twitter.com/mifos>
>
>
>
RE: Fineract banking use cases for performance/scalability evaluation
Posted by Anjil R Chinnapatlolla <an...@in.ibm.com>.
Thanks Ed for circulating it across the implementers. Will look forward for suggestions from the implementer community to add more use cases to the list.
We are just done with functional evaluation with minimal clients and accounts with below mentioned use cases and moving to a more scaled environment with increased clients, savings and loan accounts.
Will keep sharing the observations as we progress with the evaluations.
There were few issues observed, like Read Pods (Fineract Pod deployed in Read only mode) hitting Out Of Memory (due to thread limit exceeding and not actual heap) while querying for savings accounts at bulk. Will look further into the data and summarize and share observations as we start to evaluate with a scaled environment.
Thanks & Regards,
Anjil Reddy Chinnapatlolla
From: Ed Cable <ed...@mifos.org>
Date: Thursday, 2 March 2023 at 11:08 AM
To: dev@fineract.apache.org <de...@fineract.apache.org>
Subject: [EXTERNAL] Re: Fineract banking use cases for performance/scalability evaluation
Thank you Anjil for the contributions you and the rest of the team are making around performance/scalability evaluation. It will be nice to have some recent evaluations of performance beyond just for lending/credit. I will circulate your request
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
ZjQcmQRYFpfptBannerEnd
Thank you Anjil for the contributions you and the rest of the team are making around performance/scalability evaluation. It will be nice to have some recent evaluations of performance beyond just for lending/credit. I will circulate your request from other implementers across the community.
Thanks,
Ed
On Tue, Feb 28, 2023 at 11:12 AM Anjil R Chinnapatlolla <an...@in.ibm.com>> wrote:
Dear community members,
We are evaluating the performance of OCP (Openshift Container Platform) infrastructure with banking workloads on different hardware architectures. With Fineract as one of the banking workloads, we have built the load test suite with APIs covering below use cases (Majorly with Savings and Loan products).
I request the community members and teams who have Fineract deployed in production or being evaluated, to suggest specific use-cases which are more challenging in nature (in latency and scalability perspective) that we should consider adding to below list while evaluating performance of the deployment infrastructure (which in turn covers the performance of Fineract).
Setup Bank Environment
Category
Use case
Create Codes
Create client closure reason code
Create payment type code
Get payment type codes
Ledger Accounting
Create Asset Ledger Account
Create Liability Ledger Account
Create Revenue Ledger Account
Create Expense Ledger Account
List General Ledger Accounts
Financial Activity Accounting
Map Financial Activity Account
Define Savings Products
Create savings product
Define Loan Products
Create personal loan product
Setup Clients
Create client
Setup Savings Account
Submit savings account
Approve savings account
Activate savings account
Setup Loan Accounts
Submit loan application
Approve loan application
Transactions
Category
Use case
Clients
Get all clients
Deposits
Deposit to savings account
Withdrawals
Withdraw from savings account
Balance Enquiry
Balance enquiry with associations
Balance Enquiry
Post Interest
Post Interest on savings account
Disburse Loans
Disburse loan application
Undo disburse loan application
Disburse loan to savings
List All Loans
List loans
Reporting
Report savings account dormancy
Report client listing
Retrieve a report
List all reports and parameters
Thanks & Regards,
Anjil Reddy Chinnapatlolla
--
Ed Cable
President/CEO, Mifos Initiative
edcable@mifos.org<ma...@mifos.org> | Skype: edcable | Mobile: +1.484.477.8649
Collectively Creating a World of 3 Billion Maries | http://mifos.org<http://mifos.org> [https://secure.plimus.com/developers/817570/Template/icon-tiny-facebook.png] <http://facebook.com/mifos> [http://organizationsandmarkets.files.wordpress.com/2010/04/icon-tiny-twitter.png] <http://www.twitter.com/mifos>
Re: Fineract banking use cases for performance/scalability evaluation
Posted by Ed Cable <ed...@mifos.org>.
Thank you Anjil for the contributions you and the rest of the team are
making around performance/scalability evaluation. It will be nice to have
some recent evaluations of performance beyond just for lending/credit. I
will circulate your request from other implementers across the community.
Thanks,
Ed
On Tue, Feb 28, 2023 at 11:12 AM Anjil R Chinnapatlolla <an...@in.ibm.com>
wrote:
> Dear community members,
>
>
>
> We are evaluating the performance of OCP (Openshift Container Platform)
> infrastructure with banking workloads on different hardware architectures.
> With Fineract as one of the banking workloads, we have built the load test
> suite with APIs covering below use cases (Majorly with Savings and Loan
> products).
>
>
>
> I request the community members and teams who have Fineract deployed in
> production or being evaluated, to suggest specific use-cases which are more
> challenging in nature (in latency and scalability perspective) that we
> should consider adding to below list while evaluating performance of the
> deployment infrastructure (which in turn covers the performance of
> Fineract).
>
>
>
>
>
> Setup Bank Environment
>
> Category
>
> Use case
>
> *Create Codes*
>
>
>
>
>
> Create client closure reason code
>
>
>
> Create payment type code
>
>
>
> Get payment type codes
>
> *Ledger Accounting*
>
>
>
>
>
> Create Asset Ledger Account
>
>
>
> Create Liability Ledger Account
>
>
>
> Create Revenue Ledger Account
>
>
>
> Create Expense Ledger Account
>
>
>
> List General Ledger Accounts
>
> *Financial Activity Accounting*
>
>
>
>
>
> Map Financial Activity Account
>
> *Define Savings Products*
>
>
>
>
>
> Create savings product
>
> *Define Loan Products*
>
>
>
>
>
> Create personal loan product
>
> *Setup Clients*
>
>
>
>
>
> Create client
>
> *Setup Savings Account*
>
>
>
>
>
> Submit savings account
>
>
>
> Approve savings account
>
>
>
> Activate savings account
>
> *Setup Loan Accounts*
>
>
>
>
>
> Submit loan application
>
>
>
> Approve loan application
>
>
>
>
>
> *Transactions*
>
> *Category*
>
> *Use case*
>
> *Clients*
>
>
>
>
>
> Get all clients
>
> *Deposits*
>
>
>
>
>
> Deposit to savings account
>
> *Withdrawals*
>
>
>
>
>
> Withdraw from savings account
>
> *Balance Enquiry*
>
>
>
>
>
> Balance enquiry with associations
>
>
>
> Balance Enquiry
>
> *Post Interest *
>
>
>
>
>
> Post Interest on savings account
>
> *Disburse Loans*
>
>
>
>
>
> Disburse loan application
>
>
>
> Undo disburse loan application
>
>
>
> Disburse loan to savings
>
> *List All Loans*
>
>
>
>
>
> List loans
>
> *Reporting *
>
>
>
>
>
> Report savings account dormancy
>
>
>
> Report client listing
>
>
>
> Retrieve a report
>
>
>
> List all reports and parameters
>
>
>
>
>
> Thanks & Regards,
>
> Anjil Reddy Chinnapatlolla
>
--
*Ed Cable*
President/CEO, Mifos Initiative
edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
*Collectively Creating a World of 3 Billion Maries | *http://mifos.org
<http://facebook.com/mifos> <http://www.twitter.com/mifos>
Re: Fineract banking use cases for performance/scalability evaluation
Posted by John Gathogo <jo...@fiter.io>.
Hello Anjil,
From practical experience, these are the processes that strain Fineract and
would benefit greatly from performance/scalability improvements.
1. Generating general ledger report for accounts with millions of records
2. Posting accrual interest on savings accounts with millions of records
3. Posting interest on millions of savings and loans accounts
4. Posting transactions (deposit/withdrawal/interest) to a single
savings account with millions of transactions
5. Running the trial balance report when we have millions of journal
entries.
@Frank Nkuyahaga <fr...@fiter.io> can provide more low level details/advise
Thanks,
Ruhiu
On Tue, Feb 28, 2023 at 10:12 PM Anjil R Chinnapatlolla <an...@in.ibm.com>
wrote:
> Dear community members,
>
>
>
> We are evaluating the performance of OCP (Openshift Container Platform)
> infrastructure with banking workloads on different hardware architectures.
> With Fineract as one of the banking workloads, we have built the load test
> suite with APIs covering below use cases (Majorly with Savings and Loan
> products).
>
>
>
> I request the community members and teams who have Fineract deployed in
> production or being evaluated, to suggest specific use-cases which are more
> challenging in nature (in latency and scalability perspective) that we
> should consider adding to below list while evaluating performance of the
> deployment infrastructure (which in turn covers the performance of
> Fineract).
>
>
>
>
>
> Setup Bank Environment
>
> Category
>
> Use case
>
> *Create Codes*
>
>
>
>
>
> Create client closure reason code
>
>
>
> Create payment type code
>
>
>
> Get payment type codes
>
> *Ledger Accounting*
>
>
>
>
>
> Create Asset Ledger Account
>
>
>
> Create Liability Ledger Account
>
>
>
> Create Revenue Ledger Account
>
>
>
> Create Expense Ledger Account
>
>
>
> List General Ledger Accounts
>
> *Financial Activity Accounting*
>
>
>
>
>
> Map Financial Activity Account
>
> *Define Savings Products*
>
>
>
>
>
> Create savings product
>
> *Define Loan Products*
>
>
>
>
>
> Create personal loan product
>
> *Setup Clients*
>
>
>
>
>
> Create client
>
> *Setup Savings Account*
>
>
>
>
>
> Submit savings account
>
>
>
> Approve savings account
>
>
>
> Activate savings account
>
> *Setup Loan Accounts*
>
>
>
>
>
> Submit loan application
>
>
>
> Approve loan application
>
>
>
>
>
> *Transactions*
>
> *Category*
>
> *Use case*
>
> *Clients*
>
>
>
>
>
> Get all clients
>
> *Deposits*
>
>
>
>
>
> Deposit to savings account
>
> *Withdrawals*
>
>
>
>
>
> Withdraw from savings account
>
> *Balance Enquiry*
>
>
>
>
>
> Balance enquiry with associations
>
>
>
> Balance Enquiry
>
> *Post Interest *
>
>
>
>
>
> Post Interest on savings account
>
> *Disburse Loans*
>
>
>
>
>
> Disburse loan application
>
>
>
> Undo disburse loan application
>
>
>
> Disburse loan to savings
>
> *List All Loans*
>
>
>
>
>
> List loans
>
> *Reporting *
>
>
>
>
>
> Report savings account dormancy
>
>
>
> Report client listing
>
>
>
> Retrieve a report
>
>
>
> List all reports and parameters
>
>
>
>
>
> Thanks & Regards,
>
> Anjil Reddy Chinnapatlolla
>