You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kie.apache.org by "tkobayas (via GitHub)" <gi...@apache.org> on 2024/09/18 08:48:09 UTC
[PR] [incubator-kie-drools-6088] Performance issue with many kbase models … [incubator-kie-drools]
tkobayas opened a new pull request, #6089:
URL: https://github.com/apache/incubator-kie-drools/pull/6089
…and many large spreadsheets
**Issue**:
* https://github.com/apache/incubator-kie-drools/issues/6088
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [incubator-kie-drools-6088] Performance issue with many kbase models … [incubator-kie-drools]
Posted by "tkobayas (via GitHub)" <gi...@apache.org>.
tkobayas commented on PR #6089:
URL: https://github.com/apache/incubator-kie-drools/pull/6089#issuecomment-2357973952
GHA ubuntu-latest: flaky
```
2024-09-18T09:04:13.7047738Z [ERROR] Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 16.53 s <<< FAILURE! -- in org.drools.compiler.integrationtests.TimerAndCalendarFireUntilHaltTest
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [incubator-kie-drools-6088] Performance issue with many kbase models … [incubator-kie-drools]
Posted by "mariofusco (via GitHub)" <gi...@apache.org>.
mariofusco commented on PR #6089:
URL: https://github.com/apache/incubator-kie-drools/pull/6089#issuecomment-2368501442
Thanks a lot for your analysis of the problem @tkobayas. I implemented a [very similar alternative solution](https://github.com/apache/incubator-kie-drools/pull/6097), that however moves that same cache in the dtable module, trying to explain why I believe this would be a better solution. Let me know what you think about it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [incubator-kie-drools-6088] Performance issue with many kbase models … [incubator-kie-drools]
Posted by "tkobayas (via GitHub)" <gi...@apache.org>.
tkobayas commented on PR #6089:
URL: https://github.com/apache/incubator-kie-drools/pull/6089#issuecomment-2357972303
Hi @mariofusco
The performance issue only arises when a KieProject has many kbase models and many large spreadsheets.
In this case, `AbstractKieProject.buildKnowledgePackages` is called per kbase model. `AbstractKieProject.buildKnowledgePackages` calls `AbstractKieProject.addFiles` -> `KieBuilderImpl.filterFileInKBase` -> `KieBuilderImpl.filterFileInKBase` -> `KieBuilderImpl.isFileInKieBase` -> `KieBuilderImpl.packageNameForFile` -> `KieBuilderImpl.packageNameFromAsset` -> `KieBuilderImpl.packageNameFromDtable` -> `DecisionTableProviderImpl.loadFromResource` -> `SpreadsheetCompiler.compile`.
If a KieProject has 20 kbase models and 20 spreadsheets, `packageNameFromDtable` is called 400 times. One `packageNameFromDtable` may not take very long time (1 or 2 seconds), but 400 times are significant.
https://github.com/apache/incubator-kie-drools/blob/main/drools-compiler/src/main/java/org/drools/compiler/kie/builder/impl/KieBuilderImpl.java#L438
This PR introduces `packageNameCache` and brings the cache across static methods. This doesn't look nice, but firstly I wanted to share the idea.
Another possible approach would be caching the DRL generated by SpreadsheetCompiler, but where is the appropriate place to have the cache? (and when we should invalidate the cache)
---
Workaround for non-fixed version: Split KieContainer/ReleaseId per kbaseModel so you can avoid the combinatorial explosion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [incubator-kie-drools-6088] Performance issue with many kbase models … [incubator-kie-drools]
Posted by "mariofusco (via GitHub)" <gi...@apache.org>.
mariofusco commented on PR #6089:
URL: https://github.com/apache/incubator-kie-drools/pull/6089#issuecomment-2370635303
Replaced by https://github.com/apache/incubator-kie-drools/pull/6097
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] [incubator-kie-drools-6088] Performance issue with many kbase models … [incubator-kie-drools]
Posted by "mariofusco (via GitHub)" <gi...@apache.org>.
mariofusco closed pull request #6089: [incubator-kie-drools-6088] Performance issue with many kbase models …
URL: https://github.com/apache/incubator-kie-drools/pull/6089
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org