You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@fineract.apache.org by Arnold Gálovics <ga...@gmail.com> on 2022/02/04 10:32:08 UTC
Loan entity performance improvements
Hi everybody,
Wanted to bring attention to the ticket I've just created on potential
performance improvements when it comes to Loans.
https://issues.apache.org/jira/browse/FINERACT-1496
I've tried to write everything down into the ticket but let me try to sum
it up here.
Turns out the Loan entity has an EAGER relationship to the
LoanCollateralManagement entity which goes on and on and on. The full
relationship chain is Loan -> LoanCollateralManagement ->
ClientCollateralManagement -> CollateralManagementDomain ->
ApplicationCurrency
These are all eagerly loaded relationships which definitely hurts the
performance to some degree.
I'm proposing that we shall take a look at it and potentially switch from
EAGER fetching to LAZY fetching hence making sure that the data gets loaded
only when it's required.
Let me know your thoughts.
Best,
Arnold
Re: Loan entity performance improvements
Posted by Aleksandar Vidakovic <ch...@monkeysintown.com>.
... I can look into that.
On Sat, Feb 5, 2022 at 12:14 AM Ed Cable <ed...@mifos.org> wrote:
> Thanks James as I had mixed up the desired condition we want to enforce.
> Lazy loading by default as opposed to eager loading.
>
> Ed
>
> On Fri, Feb 4, 2022, 12:55 James Dailey <ja...@gmail.com> wrote:
>
>> Just to clarify, we want to only use Eager loading when absolutely
>> required , otherwise lazy loading (and avoid database locks) so we want a
>> way to test for this rule on all PRs
>>
>>
>>
>> On Fri, Feb 4, 2022 at 8:52 AM Ed Cable <ed...@mifos.org> wrote:
>>
>>> Aleks - were you still exploring introduction of Archunit
>>> https://www.archunit.org/ as you had discussed as a means of providing
>>> a way to check and enforce eager loading conditions, etc?
>>>
>>> Ed
>>>
>>> On Fri, Feb 4, 2022 at 8:33 AM Aleksandar Vidakovic <
>>> cheetah@monkeysintown.com> wrote:
>>>
>>>> ... I am currently working on 3 (integration-)test related PRs:
>>>>
>>>> - add Cucumber support to have better interaction between people
>>>> that are less involved with the implementation side, but have great domain
>>>> specific knowledge (I'll explain a bit more when it's ready)
>>>> - add JMH Microbenchmarking support; this is something that could
>>>> actually help here (collateral management) to figure out any performance
>>>> related regressions
>>>> - add Gatling support; this one would improve visibility into our
>>>> integration tests; at the moment they are only used to verify if the
>>>> business logic is still working; with Gatling we could additionally get
>>>> some hints about performance; it's not "real world" (everything runs on
>>>> localhost), but big outliers should be visible and help avoid regressions
>>>>
>>>> ... JMH Microbenchmarking in the context of the collateral management
>>>> module could really help finding any bottlenecks... and it's easy to write
>>>> and run these tests.
>>>>
>>>> I can move this up my priority list if needed... just let me know.
>>>>
>>>> Cheers,
>>>>
>>>> Aleks
>>>>
>>>> On Fri, Feb 4, 2022 at 5:02 PM Ed Cable <ed...@mifos.org> wrote:
>>>>
>>>>> Welcome to the community Arnold and thanks for documenting clearly in
>>>>> the JIRA ticket the extent of changes you would propose to ensure optimal
>>>>> performance with the loan entity.
>>>>>
>>>>> Going forward, we want to ensure necessary automated checks are in
>>>>> place for future pull requests to enforce eager versus lazy loading so we
>>>>> can catch this before a contribution is merged and have a developer address
>>>>> it accordingly. I believe Aleks is working on incorporating that into our
>>>>> review process and the CI/CD and corresponding documentation will be
>>>>> updated as those changes are made.
>>>>>
>>>>> I am tagging @Benura Abeywardena <be...@gmail.com> as he was the
>>>>> developer working on the collateral management module and hope that he has
>>>>> some time to help complete the fixes you are suggesting and proposing.
>>>>>
>>>>> Ed
>>>>>
>>>>> On Fri, Feb 4, 2022 at 5:23 AM Bharath Gowda <bg...@mifos.org> wrote:
>>>>>
>>>>>> Hi Arnold,
>>>>>>
>>>>>> Thank you for surfacing this issue.
>>>>>> Performance of the application is always an important aspect and any
>>>>>> fix related to that is always welcome and good for the application's growth
>>>>>>
>>>>>> Attaching the full Functional Documents and Video links of the
>>>>>> Collateral Module to help understand the feature while fixing the
>>>>>> Performance issue
>>>>>>
>>>>>> In Summary, Collateral Module is Divided into 3 Levels
>>>>>>
>>>>>> • Product Level
>>>>>> • Client Level
>>>>>> • Loan Level
>>>>>>
>>>>>> At the product level, the user defines the Collateral required for
>>>>>> the organization
>>>>>>
>>>>>> At the client level, the user adds the collateral which the client
>>>>>> agrees to pledge
>>>>>>
>>>>>> At the Loan level, the user attaches the collateral to the loan which
>>>>>> the client has pledged to get a loan.
>>>>>>
>>>>>> When the loan is repaid the collateral would be released from that
>>>>>> loan account and would be again available to pledge for another loan for
>>>>>> that client
>>>>>>
>>>>>>
>>>>>> Full Functional Video of the feature is available here
>>>>>> <https://drive.google.com/file/d/1r8Dxbf7ebzk4k1NR4a-rObz6LLAY3qqZ/view?usp=sharing>to
>>>>>> view
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Bharath
>>>>>> Lead Implementation Analyst | Mifos Initiative
>>>>>> Skype: live:cbharath4| Mobile: +91.7019635592
>>>>>> http://mifos.org <http://facebook.com/mifos>
>>>>>> <http://www.twitter.com/mifos>
>>>>>>
>>>>>>
>>>>>> On Fri, Feb 4, 2022 at 4:03 PM Arnold Gálovics <
>>>>>> galovicsarnold@gmail.com> wrote:
>>>>>>
>>>>>>> Hi everybody,
>>>>>>>
>>>>>>> Wanted to bring attention to the ticket I've just created on
>>>>>>> potential performance improvements when it comes to Loans.
>>>>>>> https://issues.apache.org/jira/browse/FINERACT-1496
>>>>>>>
>>>>>>> I've tried to write everything down into the ticket but let me try
>>>>>>> to sum it up here.
>>>>>>> Turns out the Loan entity has an EAGER relationship to the
>>>>>>> LoanCollateralManagement entity which goes on and on and on. The full
>>>>>>> relationship chain is Loan -> LoanCollateralManagement ->
>>>>>>> ClientCollateralManagement -> CollateralManagementDomain ->
>>>>>>> ApplicationCurrency
>>>>>>>
>>>>>>> These are all eagerly loaded relationships which definitely hurts
>>>>>>> the performance to some degree.
>>>>>>> I'm proposing that we shall take a look at it and potentially switch
>>>>>>> from EAGER fetching to LAZY fetching hence making sure that the data gets
>>>>>>> loaded only when it's required.
>>>>>>>
>>>>>>> Let me know your thoughts.
>>>>>>>
>>>>>>> Best,
>>>>>>> Arnold
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> *Ed Cable*
>>>>> President/CEO, Mifos Initiative
>>>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>>>>
>>>>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
>>>>> <http://facebook.com/mifos> <http://www.twitter.com/mifos>
>>>>>
>>>>>
>>>
>>> --
>>> *Ed Cable*
>>> President/CEO, Mifos Initiative
>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>>
>>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
>>> <http://facebook.com/mifos> <http://www.twitter.com/mifos>
>>>
>>> --
>> Sent from Gmail Mobile
>>
>
Re: Loan entity performance improvements
Posted by Ed Cable <ed...@mifos.org>.
Thanks James as I had mixed up the desired condition we want to enforce.
Lazy loading by default as opposed to eager loading.
Ed
On Fri, Feb 4, 2022, 12:55 James Dailey <ja...@gmail.com> wrote:
> Just to clarify, we want to only use Eager loading when absolutely
> required , otherwise lazy loading (and avoid database locks) so we want a
> way to test for this rule on all PRs
>
>
>
> On Fri, Feb 4, 2022 at 8:52 AM Ed Cable <ed...@mifos.org> wrote:
>
>> Aleks - were you still exploring introduction of Archunit
>> https://www.archunit.org/ as you had discussed as a means of providing a
>> way to check and enforce eager loading conditions, etc?
>>
>> Ed
>>
>> On Fri, Feb 4, 2022 at 8:33 AM Aleksandar Vidakovic <
>> cheetah@monkeysintown.com> wrote:
>>
>>> ... I am currently working on 3 (integration-)test related PRs:
>>>
>>> - add Cucumber support to have better interaction between people
>>> that are less involved with the implementation side, but have great domain
>>> specific knowledge (I'll explain a bit more when it's ready)
>>> - add JMH Microbenchmarking support; this is something that could
>>> actually help here (collateral management) to figure out any performance
>>> related regressions
>>> - add Gatling support; this one would improve visibility into our
>>> integration tests; at the moment they are only used to verify if the
>>> business logic is still working; with Gatling we could additionally get
>>> some hints about performance; it's not "real world" (everything runs on
>>> localhost), but big outliers should be visible and help avoid regressions
>>>
>>> ... JMH Microbenchmarking in the context of the collateral management
>>> module could really help finding any bottlenecks... and it's easy to write
>>> and run these tests.
>>>
>>> I can move this up my priority list if needed... just let me know.
>>>
>>> Cheers,
>>>
>>> Aleks
>>>
>>> On Fri, Feb 4, 2022 at 5:02 PM Ed Cable <ed...@mifos.org> wrote:
>>>
>>>> Welcome to the community Arnold and thanks for documenting clearly in
>>>> the JIRA ticket the extent of changes you would propose to ensure optimal
>>>> performance with the loan entity.
>>>>
>>>> Going forward, we want to ensure necessary automated checks are in
>>>> place for future pull requests to enforce eager versus lazy loading so we
>>>> can catch this before a contribution is merged and have a developer address
>>>> it accordingly. I believe Aleks is working on incorporating that into our
>>>> review process and the CI/CD and corresponding documentation will be
>>>> updated as those changes are made.
>>>>
>>>> I am tagging @Benura Abeywardena <be...@gmail.com> as he was the
>>>> developer working on the collateral management module and hope that he has
>>>> some time to help complete the fixes you are suggesting and proposing.
>>>>
>>>> Ed
>>>>
>>>> On Fri, Feb 4, 2022 at 5:23 AM Bharath Gowda <bg...@mifos.org> wrote:
>>>>
>>>>> Hi Arnold,
>>>>>
>>>>> Thank you for surfacing this issue.
>>>>> Performance of the application is always an important aspect and any
>>>>> fix related to that is always welcome and good for the application's growth
>>>>>
>>>>> Attaching the full Functional Documents and Video links of the
>>>>> Collateral Module to help understand the feature while fixing the
>>>>> Performance issue
>>>>>
>>>>> In Summary, Collateral Module is Divided into 3 Levels
>>>>>
>>>>> • Product Level
>>>>> • Client Level
>>>>> • Loan Level
>>>>>
>>>>> At the product level, the user defines the Collateral required for the
>>>>> organization
>>>>>
>>>>> At the client level, the user adds the collateral which the client
>>>>> agrees to pledge
>>>>>
>>>>> At the Loan level, the user attaches the collateral to the loan which
>>>>> the client has pledged to get a loan.
>>>>>
>>>>> When the loan is repaid the collateral would be released from that
>>>>> loan account and would be again available to pledge for another loan for
>>>>> that client
>>>>>
>>>>>
>>>>> Full Functional Video of the feature is available here
>>>>> <https://drive.google.com/file/d/1r8Dxbf7ebzk4k1NR4a-rObz6LLAY3qqZ/view?usp=sharing>to
>>>>> view
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>> Bharath
>>>>> Lead Implementation Analyst | Mifos Initiative
>>>>> Skype: live:cbharath4| Mobile: +91.7019635592
>>>>> http://mifos.org <http://facebook.com/mifos>
>>>>> <http://www.twitter.com/mifos>
>>>>>
>>>>>
>>>>> On Fri, Feb 4, 2022 at 4:03 PM Arnold Gálovics <
>>>>> galovicsarnold@gmail.com> wrote:
>>>>>
>>>>>> Hi everybody,
>>>>>>
>>>>>> Wanted to bring attention to the ticket I've just created on
>>>>>> potential performance improvements when it comes to Loans.
>>>>>> https://issues.apache.org/jira/browse/FINERACT-1496
>>>>>>
>>>>>> I've tried to write everything down into the ticket but let me try to
>>>>>> sum it up here.
>>>>>> Turns out the Loan entity has an EAGER relationship to the
>>>>>> LoanCollateralManagement entity which goes on and on and on. The full
>>>>>> relationship chain is Loan -> LoanCollateralManagement ->
>>>>>> ClientCollateralManagement -> CollateralManagementDomain ->
>>>>>> ApplicationCurrency
>>>>>>
>>>>>> These are all eagerly loaded relationships which definitely hurts the
>>>>>> performance to some degree.
>>>>>> I'm proposing that we shall take a look at it and potentially switch
>>>>>> from EAGER fetching to LAZY fetching hence making sure that the data gets
>>>>>> loaded only when it's required.
>>>>>>
>>>>>> Let me know your thoughts.
>>>>>>
>>>>>> Best,
>>>>>> Arnold
>>>>>>
>>>>>
>>>>
>>>> --
>>>> *Ed Cable*
>>>> President/CEO, Mifos Initiative
>>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>>>
>>>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
>>>> <http://facebook.com/mifos> <http://www.twitter.com/mifos>
>>>>
>>>>
>>
>> --
>> *Ed Cable*
>> President/CEO, Mifos Initiative
>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>
>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
>> <http://facebook.com/mifos> <http://www.twitter.com/mifos>
>>
>> --
> Sent from Gmail Mobile
>
Re: Loan entity performance improvements
Posted by James Dailey <ja...@gmail.com>.
Just to clarify, we want to only use Eager loading when absolutely required
, otherwise lazy loading (and avoid database locks) so we want a way to
test for this rule on all PRs
On Fri, Feb 4, 2022 at 8:52 AM Ed Cable <ed...@mifos.org> wrote:
> Aleks - were you still exploring introduction of Archunit
> https://www.archunit.org/ as you had discussed as a means of providing a
> way to check and enforce eager loading conditions, etc?
>
> Ed
>
> On Fri, Feb 4, 2022 at 8:33 AM Aleksandar Vidakovic <
> cheetah@monkeysintown.com> wrote:
>
>> ... I am currently working on 3 (integration-)test related PRs:
>>
>> - add Cucumber support to have better interaction between people that
>> are less involved with the implementation side, but have great domain
>> specific knowledge (I'll explain a bit more when it's ready)
>> - add JMH Microbenchmarking support; this is something that could
>> actually help here (collateral management) to figure out any performance
>> related regressions
>> - add Gatling support; this one would improve visibility into our
>> integration tests; at the moment they are only used to verify if the
>> business logic is still working; with Gatling we could additionally get
>> some hints about performance; it's not "real world" (everything runs on
>> localhost), but big outliers should be visible and help avoid regressions
>>
>> ... JMH Microbenchmarking in the context of the collateral management
>> module could really help finding any bottlenecks... and it's easy to write
>> and run these tests.
>>
>> I can move this up my priority list if needed... just let me know.
>>
>> Cheers,
>>
>> Aleks
>>
>> On Fri, Feb 4, 2022 at 5:02 PM Ed Cable <ed...@mifos.org> wrote:
>>
>>> Welcome to the community Arnold and thanks for documenting clearly in
>>> the JIRA ticket the extent of changes you would propose to ensure optimal
>>> performance with the loan entity.
>>>
>>> Going forward, we want to ensure necessary automated checks are in place
>>> for future pull requests to enforce eager versus lazy loading so we can
>>> catch this before a contribution is merged and have a developer address it
>>> accordingly. I believe Aleks is working on incorporating that into our
>>> review process and the CI/CD and corresponding documentation will be
>>> updated as those changes are made.
>>>
>>> I am tagging @Benura Abeywardena <be...@gmail.com> as he was the
>>> developer working on the collateral management module and hope that he has
>>> some time to help complete the fixes you are suggesting and proposing.
>>>
>>> Ed
>>>
>>> On Fri, Feb 4, 2022 at 5:23 AM Bharath Gowda <bg...@mifos.org> wrote:
>>>
>>>> Hi Arnold,
>>>>
>>>> Thank you for surfacing this issue.
>>>> Performance of the application is always an important aspect and any
>>>> fix related to that is always welcome and good for the application's growth
>>>>
>>>> Attaching the full Functional Documents and Video links of the
>>>> Collateral Module to help understand the feature while fixing the
>>>> Performance issue
>>>>
>>>> In Summary, Collateral Module is Divided into 3 Levels
>>>>
>>>> • Product Level
>>>> • Client Level
>>>> • Loan Level
>>>>
>>>> At the product level, the user defines the Collateral required for the
>>>> organization
>>>>
>>>> At the client level, the user adds the collateral which the client
>>>> agrees to pledge
>>>>
>>>> At the Loan level, the user attaches the collateral to the loan which
>>>> the client has pledged to get a loan.
>>>>
>>>> When the loan is repaid the collateral would be released from that loan
>>>> account and would be again available to pledge for another loan for that
>>>> client
>>>>
>>>>
>>>> Full Functional Video of the feature is available here
>>>> <https://drive.google.com/file/d/1r8Dxbf7ebzk4k1NR4a-rObz6LLAY3qqZ/view?usp=sharing>to
>>>> view
>>>>
>>>>
>>>>
>>>> Regards,
>>>> Bharath
>>>> Lead Implementation Analyst | Mifos Initiative
>>>> Skype: live:cbharath4| Mobile: +91.7019635592
>>>> http://mifos.org <http://facebook.com/mifos>
>>>> <http://www.twitter.com/mifos>
>>>>
>>>>
>>>> On Fri, Feb 4, 2022 at 4:03 PM Arnold Gálovics <
>>>> galovicsarnold@gmail.com> wrote:
>>>>
>>>>> Hi everybody,
>>>>>
>>>>> Wanted to bring attention to the ticket I've just created on potential
>>>>> performance improvements when it comes to Loans.
>>>>> https://issues.apache.org/jira/browse/FINERACT-1496
>>>>>
>>>>> I've tried to write everything down into the ticket but let me try to
>>>>> sum it up here.
>>>>> Turns out the Loan entity has an EAGER relationship to the
>>>>> LoanCollateralManagement entity which goes on and on and on. The full
>>>>> relationship chain is Loan -> LoanCollateralManagement ->
>>>>> ClientCollateralManagement -> CollateralManagementDomain ->
>>>>> ApplicationCurrency
>>>>>
>>>>> These are all eagerly loaded relationships which definitely hurts the
>>>>> performance to some degree.
>>>>> I'm proposing that we shall take a look at it and potentially switch
>>>>> from EAGER fetching to LAZY fetching hence making sure that the data gets
>>>>> loaded only when it's required.
>>>>>
>>>>> Let me know your thoughts.
>>>>>
>>>>> Best,
>>>>> Arnold
>>>>>
>>>>
>>>
>>> --
>>> *Ed Cable*
>>> President/CEO, Mifos Initiative
>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>>
>>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
>>> <http://facebook.com/mifos> <http://www.twitter.com/mifos>
>>>
>>>
>
> --
> *Ed Cable*
> President/CEO, Mifos Initiative
> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>
> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
> <http://facebook.com/mifos> <http://www.twitter.com/mifos>
>
> --
Sent from Gmail Mobile
Re: Loan entity performance improvements
Posted by Ed Cable <ed...@mifos.org>.
Aleks - were you still exploring introduction of Archunit
https://www.archunit.org/ as you had discussed as a means of providing a
way to check and enforce eager loading conditions, etc?
Ed
On Fri, Feb 4, 2022 at 8:33 AM Aleksandar Vidakovic <
cheetah@monkeysintown.com> wrote:
> ... I am currently working on 3 (integration-)test related PRs:
>
> - add Cucumber support to have better interaction between people that
> are less involved with the implementation side, but have great domain
> specific knowledge (I'll explain a bit more when it's ready)
> - add JMH Microbenchmarking support; this is something that could
> actually help here (collateral management) to figure out any performance
> related regressions
> - add Gatling support; this one would improve visibility into our
> integration tests; at the moment they are only used to verify if the
> business logic is still working; with Gatling we could additionally get
> some hints about performance; it's not "real world" (everything runs on
> localhost), but big outliers should be visible and help avoid regressions
>
> ... JMH Microbenchmarking in the context of the collateral management
> module could really help finding any bottlenecks... and it's easy to write
> and run these tests.
>
> I can move this up my priority list if needed... just let me know.
>
> Cheers,
>
> Aleks
>
> On Fri, Feb 4, 2022 at 5:02 PM Ed Cable <ed...@mifos.org> wrote:
>
>> Welcome to the community Arnold and thanks for documenting clearly in the
>> JIRA ticket the extent of changes you would propose to ensure optimal
>> performance with the loan entity.
>>
>> Going forward, we want to ensure necessary automated checks are in place
>> for future pull requests to enforce eager versus lazy loading so we can
>> catch this before a contribution is merged and have a developer address it
>> accordingly. I believe Aleks is working on incorporating that into our
>> review process and the CI/CD and corresponding documentation will be
>> updated as those changes are made.
>>
>> I am tagging @Benura Abeywardena <be...@gmail.com> as he was the
>> developer working on the collateral management module and hope that he has
>> some time to help complete the fixes you are suggesting and proposing.
>>
>> Ed
>>
>> On Fri, Feb 4, 2022 at 5:23 AM Bharath Gowda <bg...@mifos.org> wrote:
>>
>>> Hi Arnold,
>>>
>>> Thank you for surfacing this issue.
>>> Performance of the application is always an important aspect and any fix
>>> related to that is always welcome and good for the application's growth
>>>
>>> Attaching the full Functional Documents and Video links of the
>>> Collateral Module to help understand the feature while fixing the
>>> Performance issue
>>>
>>> In Summary, Collateral Module is Divided into 3 Levels
>>>
>>> • Product Level
>>> • Client Level
>>> • Loan Level
>>>
>>> At the product level, the user defines the Collateral required for the
>>> organization
>>>
>>> At the client level, the user adds the collateral which the client
>>> agrees to pledge
>>>
>>> At the Loan level, the user attaches the collateral to the loan which
>>> the client has pledged to get a loan.
>>>
>>> When the loan is repaid the collateral would be released from that loan
>>> account and would be again available to pledge for another loan for that
>>> client
>>>
>>>
>>> Full Functional Video of the feature is available here
>>> <https://drive.google.com/file/d/1r8Dxbf7ebzk4k1NR4a-rObz6LLAY3qqZ/view?usp=sharing>to
>>> view
>>>
>>>
>>>
>>> Regards,
>>> Bharath
>>> Lead Implementation Analyst | Mifos Initiative
>>> Skype: live:cbharath4| Mobile: +91.7019635592
>>> http://mifos.org <http://facebook.com/mifos>
>>> <http://www.twitter.com/mifos>
>>>
>>>
>>> On Fri, Feb 4, 2022 at 4:03 PM Arnold Gálovics <ga...@gmail.com>
>>> wrote:
>>>
>>>> Hi everybody,
>>>>
>>>> Wanted to bring attention to the ticket I've just created on potential
>>>> performance improvements when it comes to Loans.
>>>> https://issues.apache.org/jira/browse/FINERACT-1496
>>>>
>>>> I've tried to write everything down into the ticket but let me try to
>>>> sum it up here.
>>>> Turns out the Loan entity has an EAGER relationship to the
>>>> LoanCollateralManagement entity which goes on and on and on. The full
>>>> relationship chain is Loan -> LoanCollateralManagement ->
>>>> ClientCollateralManagement -> CollateralManagementDomain ->
>>>> ApplicationCurrency
>>>>
>>>> These are all eagerly loaded relationships which definitely hurts the
>>>> performance to some degree.
>>>> I'm proposing that we shall take a look at it and potentially switch
>>>> from EAGER fetching to LAZY fetching hence making sure that the data gets
>>>> loaded only when it's required.
>>>>
>>>> Let me know your thoughts.
>>>>
>>>> Best,
>>>> Arnold
>>>>
>>>
>>
>> --
>> *Ed Cable*
>> President/CEO, Mifos Initiative
>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>
>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
>> <http://facebook.com/mifos> <http://www.twitter.com/mifos>
>>
>>
--
*Ed Cable*
President/CEO, Mifos Initiative
edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
*Collectively Creating a World of 3 Billion Maries | *http://mifos.org
<http://facebook.com/mifos> <http://www.twitter.com/mifos>
Re: Loan entity performance improvements
Posted by Aleksandar Vidakovic <ch...@monkeysintown.com>.
... I am currently working on 3 (integration-)test related PRs:
- add Cucumber support to have better interaction between people that
are less involved with the implementation side, but have great domain
specific knowledge (I'll explain a bit more when it's ready)
- add JMH Microbenchmarking support; this is something that could
actually help here (collateral management) to figure out any performance
related regressions
- add Gatling support; this one would improve visibility into our
integration tests; at the moment they are only used to verify if the
business logic is still working; with Gatling we could additionally get
some hints about performance; it's not "real world" (everything runs on
localhost), but big outliers should be visible and help avoid regressions
... JMH Microbenchmarking in the context of the collateral management
module could really help finding any bottlenecks... and it's easy to write
and run these tests.
I can move this up my priority list if needed... just let me know.
Cheers,
Aleks
On Fri, Feb 4, 2022 at 5:02 PM Ed Cable <ed...@mifos.org> wrote:
> Welcome to the community Arnold and thanks for documenting clearly in the
> JIRA ticket the extent of changes you would propose to ensure optimal
> performance with the loan entity.
>
> Going forward, we want to ensure necessary automated checks are in place
> for future pull requests to enforce eager versus lazy loading so we can
> catch this before a contribution is merged and have a developer address it
> accordingly. I believe Aleks is working on incorporating that into our
> review process and the CI/CD and corresponding documentation will be
> updated as those changes are made.
>
> I am tagging @Benura Abeywardena <be...@gmail.com> as he was the
> developer working on the collateral management module and hope that he has
> some time to help complete the fixes you are suggesting and proposing.
>
> Ed
>
> On Fri, Feb 4, 2022 at 5:23 AM Bharath Gowda <bg...@mifos.org> wrote:
>
>> Hi Arnold,
>>
>> Thank you for surfacing this issue.
>> Performance of the application is always an important aspect and any fix
>> related to that is always welcome and good for the application's growth
>>
>> Attaching the full Functional Documents and Video links of the Collateral
>> Module to help understand the feature while fixing the Performance issue
>>
>> In Summary, Collateral Module is Divided into 3 Levels
>>
>> • Product Level
>> • Client Level
>> • Loan Level
>>
>> At the product level, the user defines the Collateral required for the
>> organization
>>
>> At the client level, the user adds the collateral which the client agrees
>> to pledge
>>
>> At the Loan level, the user attaches the collateral to the loan which the
>> client has pledged to get a loan.
>>
>> When the loan is repaid the collateral would be released from that loan
>> account and would be again available to pledge for another loan for that
>> client
>>
>>
>> Full Functional Video of the feature is available here
>> <https://drive.google.com/file/d/1r8Dxbf7ebzk4k1NR4a-rObz6LLAY3qqZ/view?usp=sharing>to
>> view
>>
>>
>>
>> Regards,
>> Bharath
>> Lead Implementation Analyst | Mifos Initiative
>> Skype: live:cbharath4| Mobile: +91.7019635592
>> http://mifos.org <http://facebook.com/mifos>
>> <http://www.twitter.com/mifos>
>>
>>
>> On Fri, Feb 4, 2022 at 4:03 PM Arnold Gálovics <ga...@gmail.com>
>> wrote:
>>
>>> Hi everybody,
>>>
>>> Wanted to bring attention to the ticket I've just created on potential
>>> performance improvements when it comes to Loans.
>>> https://issues.apache.org/jira/browse/FINERACT-1496
>>>
>>> I've tried to write everything down into the ticket but let me try to
>>> sum it up here.
>>> Turns out the Loan entity has an EAGER relationship to the
>>> LoanCollateralManagement entity which goes on and on and on. The full
>>> relationship chain is Loan -> LoanCollateralManagement ->
>>> ClientCollateralManagement -> CollateralManagementDomain ->
>>> ApplicationCurrency
>>>
>>> These are all eagerly loaded relationships which definitely hurts the
>>> performance to some degree.
>>> I'm proposing that we shall take a look at it and potentially switch
>>> from EAGER fetching to LAZY fetching hence making sure that the data gets
>>> loaded only when it's required.
>>>
>>> Let me know your thoughts.
>>>
>>> Best,
>>> Arnold
>>>
>>
>
> --
> *Ed Cable*
> President/CEO, Mifos Initiative
> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>
> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
> <http://facebook.com/mifos> <http://www.twitter.com/mifos>
>
>
Re: Loan entity performance improvements
Posted by Ed Cable <ed...@mifos.org>.
Welcome to the community Arnold and thanks for documenting clearly in the
JIRA ticket the extent of changes you would propose to ensure optimal
performance with the loan entity.
Going forward, we want to ensure necessary automated checks are in place
for future pull requests to enforce eager versus lazy loading so we can
catch this before a contribution is merged and have a developer address it
accordingly. I believe Aleks is working on incorporating that into our
review process and the CI/CD and corresponding documentation will be
updated as those changes are made.
I am tagging @Benura Abeywardena <be...@gmail.com> as he was the
developer working on the collateral management module and hope that he has
some time to help complete the fixes you are suggesting and proposing.
Ed
On Fri, Feb 4, 2022 at 5:23 AM Bharath Gowda <bg...@mifos.org> wrote:
> Hi Arnold,
>
> Thank you for surfacing this issue.
> Performance of the application is always an important aspect and any fix
> related to that is always welcome and good for the application's growth
>
> Attaching the full Functional Documents and Video links of the Collateral
> Module to help understand the feature while fixing the Performance issue
>
> In Summary, Collateral Module is Divided into 3 Levels
>
> • Product Level
> • Client Level
> • Loan Level
>
> At the product level, the user defines the Collateral required for the
> organization
>
> At the client level, the user adds the collateral which the client agrees
> to pledge
>
> At the Loan level, the user attaches the collateral to the loan which the
> client has pledged to get a loan.
>
> When the loan is repaid the collateral would be released from that loan
> account and would be again available to pledge for another loan for that
> client
>
>
> Full Functional Video of the feature is available here
> <https://drive.google.com/file/d/1r8Dxbf7ebzk4k1NR4a-rObz6LLAY3qqZ/view?usp=sharing>to
> view
>
>
>
> Regards,
> Bharath
> Lead Implementation Analyst | Mifos Initiative
> Skype: live:cbharath4| Mobile: +91.7019635592
> http://mifos.org <http://facebook.com/mifos>
> <http://www.twitter.com/mifos>
>
>
> On Fri, Feb 4, 2022 at 4:03 PM Arnold Gálovics <ga...@gmail.com>
> wrote:
>
>> Hi everybody,
>>
>> Wanted to bring attention to the ticket I've just created on potential
>> performance improvements when it comes to Loans.
>> https://issues.apache.org/jira/browse/FINERACT-1496
>>
>> I've tried to write everything down into the ticket but let me try to sum
>> it up here.
>> Turns out the Loan entity has an EAGER relationship to the
>> LoanCollateralManagement entity which goes on and on and on. The full
>> relationship chain is Loan -> LoanCollateralManagement ->
>> ClientCollateralManagement -> CollateralManagementDomain ->
>> ApplicationCurrency
>>
>> These are all eagerly loaded relationships which definitely hurts the
>> performance to some degree.
>> I'm proposing that we shall take a look at it and potentially switch from
>> EAGER fetching to LAZY fetching hence making sure that the data gets loaded
>> only when it's required.
>>
>> Let me know your thoughts.
>>
>> Best,
>> Arnold
>>
>
--
*Ed Cable*
President/CEO, Mifos Initiative
edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
*Collectively Creating a World of 3 Billion Maries | *http://mifos.org
<http://facebook.com/mifos> <http://www.twitter.com/mifos>
Re: Loan entity performance improvements
Posted by Bharath Gowda <bg...@mifos.org>.
Hi Arnold,
Thank you for surfacing this issue.
Performance of the application is always an important aspect and any fix
related to that is always welcome and good for the application's growth
Attaching the full Functional Documents and Video links of the Collateral
Module to help understand the feature while fixing the Performance issue
In Summary, Collateral Module is Divided into 3 Levels
• Product Level
• Client Level
• Loan Level
At the product level, the user defines the Collateral required for the
organization
At the client level, the user adds the collateral which the client agrees
to pledge
At the Loan level, the user attaches the collateral to the loan which the
client has pledged to get a loan.
When the loan is repaid the collateral would be released from that loan
account and would be again available to pledge for another loan for that
client
Full Functional Video of the feature is available here
<https://drive.google.com/file/d/1r8Dxbf7ebzk4k1NR4a-rObz6LLAY3qqZ/view?usp=sharing>to
view
Regards,
Bharath
Lead Implementation Analyst | Mifos Initiative
Skype: live:cbharath4| Mobile: +91.7019635592
http://mifos.org <http://facebook.com/mifos>
<http://www.twitter.com/mifos>
On Fri, Feb 4, 2022 at 4:03 PM Arnold Gálovics <ga...@gmail.com>
wrote:
> Hi everybody,
>
> Wanted to bring attention to the ticket I've just created on potential
> performance improvements when it comes to Loans.
> https://issues.apache.org/jira/browse/FINERACT-1496
>
> I've tried to write everything down into the ticket but let me try to sum
> it up here.
> Turns out the Loan entity has an EAGER relationship to the
> LoanCollateralManagement entity which goes on and on and on. The full
> relationship chain is Loan -> LoanCollateralManagement ->
> ClientCollateralManagement -> CollateralManagementDomain ->
> ApplicationCurrency
>
> These are all eagerly loaded relationships which definitely hurts the
> performance to some degree.
> I'm proposing that we shall take a look at it and potentially switch from
> EAGER fetching to LAZY fetching hence making sure that the data gets loaded
> only when it's required.
>
> Let me know your thoughts.
>
> Best,
> Arnold
>