Tweaking the "Execute standing instructions" handle batches instead of the whole lump-some of instructions

[Kigred Developer <ki...@gmail.com>]

Hello Devs,

You find that when groups are starting financial operations with small
number of transactions (both loans and savings) and the minimum machine
specs will work fine for sometime. This changes with time as the number of
transactions grows and overnight jobs hit out-of-memory issues.

Is it a good idea to tweak a job such that it approaches the task
batch-wise? For example you find the an 8GB machine will process 1000
standing instructions without a problem but run into memory exceptions when
the number of instructions changes to 1200. So it a good idea to tweak the
job such it handles the 1000 to 1200 instructions in batches of say 250
instructions, then another 250 etc? With the same machine job will handle
task without running into memory issues.

Is this the way it should work? What is the other way (without changing
server specs)?

Regards.
Wilfred

Re: Tweaking the "Execute standing instructions" handle batches instead of the whole lump-some of instructions

[Kigred Developer <ki...@gmail.com>]

Hello Bharath,

Thanks for your reply.
I am using MariaDB on a standalone server.

The figures I used in the original email are hypothetical. I basically want
to validate if handling these standing instructions can be a long term
solution for this particular job.

Regards.
Wilfred

On Wed, 10 Sept 2025, 16:45 Bharath Gowda, <bg...@mifos.org> wrote:

> Hi Wilfred,
>
> I understand that you do not want to change the server specs but which
> Database are you using and is that a standalone server?
>
> Few options we could consider first
> 1. Having a standalone server for DB
> 2. Check and tweek the DB settings and configurations
> 3. Small changes to jobs configs
>      - Disable all unnecessary jobs as per your functionality usage
>      - Check and Adjust the cron timing of each active job to make sure
> one job runs at a time
>
> Tweaking the jobs to process in batches could also work but eventually
> this problem will occur for other jobs or other actions when the volume is
> again increased in future.
> So, I would suggest finding a permanent fix for the issue.
>
>
>
> Regards,
> Bharath
> Lead Implementation Analyst | Mifos Initiative
> PMC Member | Apache Fineract
> Mobile: +91.7019635592
> http://mifos.org  <http://facebook.com/mifos>
> <http://www.twitter.com/mifos>
>
>
> On Wed, Sep 10, 2025 at 6:46 PM Kigred Developer <
> kigred.developer@gmail.com> wrote:
>
>> Hello Devs,
>>
>> You find that when groups are starting financial operations with small
>> number of transactions (both loans and savings) and the minimum machine
>> specs will work fine for sometime. This changes with time as the number of
>> transactions grows and overnight jobs hit out-of-memory issues.
>>
>> Is it a good idea to tweak a job such that it approaches the task
>> batch-wise? For example you find the an 8GB machine will process 1000
>> standing instructions without a problem but run into memory exceptions when
>> the number of instructions changes to 1200. So it a good idea to tweak the
>> job such it handles the 1000 to 1200 instructions in batches of say 250
>> instructions, then another 250 etc? With the same machine job will handle
>> task without running into memory issues.
>>
>> Is this the way it should work? What is the other way (without changing
>> server specs)?
>>
>> Regards.
>> Wilfred
>>
>

Re: Tweaking the "Execute standing instructions" handle batches instead of the whole lump-some of instructions

[Bharath Gowda <bg...@mifos.org>]

Hi Wilfred,

I understand that you do not want to change the server specs but which
Database are you using and is that a standalone server?

Few options we could consider first
1. Having a standalone server for DB
2. Check and tweek the DB settings and configurations
3. Small changes to jobs configs
     - Disable all unnecessary jobs as per your functionality usage
     - Check and Adjust the cron timing of each active job to make sure one
job runs at a time

Tweaking the jobs to process in batches could also work but eventually this
problem will occur for other jobs or other actions when the volume is again
increased in future.
So, I would suggest finding a permanent fix for the issue.



Regards,
Bharath
Lead Implementation Analyst | Mifos Initiative
PMC Member | Apache Fineract
Mobile: +91.7019635592
http://mifos.org  <http://facebook.com/mifos>
<http://www.twitter.com/mifos>


On Wed, Sep 10, 2025 at 6:46 PM Kigred Developer <ki...@gmail.com>
wrote:

> Hello Devs,
>
> You find that when groups are starting financial operations with small
> number of transactions (both loans and savings) and the minimum machine
> specs will work fine for sometime. This changes with time as the number of
> transactions grows and overnight jobs hit out-of-memory issues.
>
> Is it a good idea to tweak a job such that it approaches the task
> batch-wise? For example you find the an 8GB machine will process 1000
> standing instructions without a problem but run into memory exceptions when
> the number of instructions changes to 1200. So it a good idea to tweak the
> job such it handles the 1000 to 1200 instructions in batches of say 250
> instructions, then another 250 etc? With the same machine job will handle
> task without running into memory issues.
>
> Is this the way it should work? What is the other way (without changing
> server specs)?
>
> Regards.
> Wilfred
>

Re: Tweaking the "Execute standing instructions" handle batches instead of the whole lump-some of instructions

[Paul <pc...@gmail.com>]

*From a business POV.  *
*My Free Business Advice Has Double Money Back Warranty!** 🤪*

Even very small businesses should make *scale friendly* decisions. Always
position towards account growth rather than using constraints when
possible.  Have a response plan BEFORE it happens.

In the use case mentioned:

*Hard* metal self hosted, increase memory, its a cheap, one time expense.

Writing constraints often has unintended consequences elsewhere . . .  and
actually costs more than the memory.

A simple business promotion can increase accounts rapidly and then
customers and employees are left waiting while the server juggles
insufficient memory.

*Cloud*, maybe use a constraint until the volume increases to a
defined plateau where the cost of upscaling memory as a recurring cost is
justified.


Maintain awareness of Tech Debt and have a simple mitigation plan to
constantly think towards growth.
If allowing growth without restraint or interruption *= *the ability to
scale, THEN constraints which artificially lower performance are contra to
those goals.
Never intentionally degrade your capacity UNLESS its an absolute MUST.

--
Paul

Re: Tweaking the "Execute standing instructions" handle batches instead of the whole lump-some of instructions

[VICTOR MANUEL ROMERO RODRIGUEZ <vi...@fintecheando.mx>]

Kigred,

Another possible solution base on the error "out-of-memory"  is to set the
value of the Java Virtual Machine heap maximum size, also make sure it is
not limited and depending on your deployment the value must be set in the
docker compose, the K8s descriptor, the Tomcat environment file, etc.

If you set this value for your Apache Fineract instance (this is ONLY an
example):

JAVA_TOOL_OPTIONS="-Xmx6G"

It means “limit Apache Fineract heap size to 6 GB.”

Try to use different values not greater than the available memory of your
system. And use different workloads. This process is called fine tuning.

Regards

Victor

El mié, 10 sept 2025 a las 7:16, Kigred Developer (<
kigred.developer@gmail.com>) escribió:

> Hello Devs,
>
> You find that when groups are starting financial operations with small
> number of transactions (both loans and savings) and the minimum machine
> specs will work fine for sometime. This changes with time as the number of
> transactions grows and overnight jobs hit out-of-memory issues.
>
> Is it a good idea to tweak a job such that it approaches the task
> batch-wise? For example you find the an 8GB machine will process 1000
> standing instructions without a problem but run into memory exceptions when
> the number of instructions changes to 1200. So it a good idea to tweak the
> job such it handles the 1000 to 1200 instructions in batches of say 250
> instructions, then another 250 etc? With the same machine job will handle
> task without running into memory issues.
>
> Is this the way it should work? What is the other way (without changing
> server specs)?
>
> Regards.
> Wilfred
>