You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@fineract.apache.org by "Avik Ganguly (JIRA)" <ji...@apache.org> on 2017/04/18 06:38:41 UTC
[jira] [Updated] (FINERACT-428) Parallelization of Jobs
[ https://issues.apache.org/jira/browse/FINERACT-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avik Ganguly updated FINERACT-428:
----------------------------------
Description:
For starters, it will be useful to have some technical configuration added as job parameter for each of the below jobs, that is batch size and thread pool size for executor service so this has a dependency on FINERACT-425.
Add migration script to add those 2 configurations to each of the following jobs with the value of batch size being 500 and thread pool size being 16 :-
Add Accrual Transactions
Add Periodic Accrual Transactions
Add Accrual Transactions For Loans With Income Posted As Transactions
Generate Loan Loss Provisioning
Post Interest for Savings
Update Loan Summary
This would require separation of core functionality to a separate class.
Simplified example :-
```
final ExecutorService executor = Executors.newFixedThreadPool(threadPoolSize);
final LocalDate dueDate = DateUtils.getLocalDateOfTenant();
final Collection<LoanAccountData> loansToBeRepaidData = this.loanReadPlatformService
.retrieveLoansToBeRepaidFromAdvancePayment(dueDate);
Iterable<List<LoanAccountData>> loansToBeRepaid = Iterables.partition(loansToBeRepaidData, batchSize);
final Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
List<Callable<Object>> advancePaymentPosters = new ArrayList<Callable<Object>>();
for (List<LoanAccountData> subList : loansToBeRepaid) {
AdvancePaymentPoster poster = (AdvancePaymentPoster) this.applicationContext.getBean("advancePaymentPoster");
poster.setLoans(subList);
poster.setTenant(ThreadLocalContextUtil.getTenant());
poster.setAuthentication(authentication);
poster.setCommandService(commandService);
advancePaymentPosters.add(Executors.callable(poster));
}
try {
List<Future<Object>> responses = executor.invokeAll(advancePaymentPosters);
for(Future f : responses) {
f.get();
}
} catch (InterruptedException e1) {
e1.printStackTrace();
}
executor.shutdown();
```
was:
For starters, it will be useful to have some technical configuration added as job parameter for each of the below jobs, that is batch size and thread pool size for executor service so this has a dependency on FINERACT-425.
Add migration script to add those 2 configurations to each of the following jobs with the value of batch size being 500 and thread pool size being 16 :-
Add Accrual Transactions
Add Periodic Accrual Transactions
Add Accrual Transactions For Loans With Income Posted As Transactions
Generate Loan Loss Provisioning
Post Interest for Savings
Update Loan Summary
This would require separation of core functionality to a separate class.
Simplified example :-
final ExecutorService executor = Executors.newFixedThreadPool(threadPoolSize);
final LocalDate dueDate = DateUtils.getLocalDateOfTenant();
final Collection<LoanAccountData> loansToBeRepaidData = this.loanReadPlatformService
.retrieveLoansToBeRepaidFromAdvancePayment(dueDate);
Iterable<List<LoanAccountData>> loansToBeRepaid = Iterables.partition(loansToBeRepaidData, batchSize);
final Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
List<Callable<Object>> advancePaymentPosters = new ArrayList<Callable<Object>>();
for (List<LoanAccountData> subList : loansToBeRepaid) {
AdvancePaymentPoster poster = (AdvancePaymentPoster) this.applicationContext.getBean("advancePaymentPoster");
poster.setLoans(subList);
poster.setTenant(ThreadLocalContextUtil.getTenant());
poster.setAuthentication(authentication);
poster.setCommandService(commandService);
advancePaymentPosters.add(Executors.callable(poster));
}
try {
List<Future<Object>> responses = executor.invokeAll(advancePaymentPosters);
for(Future f : responses) {
f.get();
}
} catch (InterruptedException e1) {
e1.printStackTrace();
}
executor.shutdown();
> Parallelization of Jobs
> -----------------------
>
> Key: FINERACT-428
> URL: https://issues.apache.org/jira/browse/FINERACT-428
> Project: Apache Fineract
> Issue Type: Improvement
> Components: Loan, Savings
> Reporter: Avik Ganguly
> Assignee: Markus Geiss
> Labels: performance
>
> For starters, it will be useful to have some technical configuration added as job parameter for each of the below jobs, that is batch size and thread pool size for executor service so this has a dependency on FINERACT-425.
> Add migration script to add those 2 configurations to each of the following jobs with the value of batch size being 500 and thread pool size being 16 :-
> Add Accrual Transactions
> Add Periodic Accrual Transactions
> Add Accrual Transactions For Loans With Income Posted As Transactions
> Generate Loan Loss Provisioning
> Post Interest for Savings
> Update Loan Summary
> This would require separation of core functionality to a separate class.
> Simplified example :-
> ```
> final ExecutorService executor = Executors.newFixedThreadPool(threadPoolSize);
>
> final LocalDate dueDate = DateUtils.getLocalDateOfTenant();
> final Collection<LoanAccountData> loansToBeRepaidData = this.loanReadPlatformService
> .retrieveLoansToBeRepaidFromAdvancePayment(dueDate);
> Iterable<List<LoanAccountData>> loansToBeRepaid = Iterables.partition(loansToBeRepaidData, batchSize);
> final Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
> List<Callable<Object>> advancePaymentPosters = new ArrayList<Callable<Object>>();
>
> for (List<LoanAccountData> subList : loansToBeRepaid) {
> AdvancePaymentPoster poster = (AdvancePaymentPoster) this.applicationContext.getBean("advancePaymentPoster");
> poster.setLoans(subList);
> poster.setTenant(ThreadLocalContextUtil.getTenant());
> poster.setAuthentication(authentication);
> poster.setCommandService(commandService);
> advancePaymentPosters.add(Executors.callable(poster));
> }
> try {
> List<Future<Object>> responses = executor.invokeAll(advancePaymentPosters);
> for(Future f : responses) {
> f.get();
> }
> } catch (InterruptedException e1) {
> e1.printStackTrace();
> }
>
> executor.shutdown();
> ```
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)