You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/02/29 15:44:23 UTC
[GitHub] [druid] calvinhkf opened a new issue #9442: Inconsistent library
versions notice.
calvinhkf opened a new issue #9442: Inconsistent library versions notice.
URL: https://github.com/apache/druid/issues/9442
Hi. I have implemented a tool to detect library version inconsistencies. Your project have 6 inconsistent libraries and 2 false consistent libraries.
Take org.apache.hive:hive-storage-api for example, this library is declared as version 2.6.0 in extensions-core/orc-extensions, 2.7.0 in extensions-core/druid-bloom-filter and etc... Such version inconsistencies may cause unnecessary maintenance effort in the long run. For example, if two modules become inter-dependent, library version conflict may happen. It has already become a common issue and hinders development progress. Thus a version harmonization is necessary.
Provided we applied a version harmonization, I calculated the cost it may have to harmonize to all upper versions including an up-to-date one. The cost refers to POM config changes and API invocation changes. Take org.apache.hive:hive-storage-api for example, if we harmonize all the library versions into 2.7.0. The concern is, how much should the project code adapt to the newer library version. We list an effort table to quantify the harmonization cost.
The effort table is listed below. It shows the overall harmonization effort by modules. The columns represents the number of library APIs and API calls(NA,NAC), deleted APIs and API calls(NDA,NDAC) as well as modified API and API calls(NMA,NMAC). Modified APIs refers to those APIs whose call graph is not the same as previous version. Take the first row for example, if upgrading the library into version 2.7.0. Given that 3 APIs is used in module extensions-core/druid-bloom-filter, 0 of them is deleted in a recommended version(which will throw a NoMethodFoundError unless re-compiling the project), 0 of them is regarded as modified which could break the former API contract.
|Index|Module|NA(NAC)|NDA(NDAC)|NMA(NMAC)|
|-|-|-|-|-|
|1|extensions-core/druid-bloom-filter|3(9)|0(0)|0(0)|
|2|extensions-core/orc-extensions|4(4)|0(0)|0(0)|
Also we provided another table to show the potential files that may be affected due to library API change, which could help to spot the concerned API usage and rerun the test cases.
As for false consistency, take com.google.guava guava jar for example. The library is declared in version 16.0.1 in all modules. However they are declared differently. As components are developed in parallel, if one single library version is updated, which could become inconsistent as mentioned above, may cause above-mentioned inconsistency issues
If you are interested, you can have a more complete and detailed report in the attached PDF file.
[druid-io druid.pdf](https://github.com/apache/druid/files/4270521/druid-io.druid.pdf)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] ccaominh commented on issue #9442: Inconsistent library
versions notice.
Posted by GitBox <gi...@apache.org>.
ccaominh commented on issue #9442: Inconsistent library versions notice.
URL: https://github.com/apache/druid/issues/9442#issuecomment-593029911
@calvinhkf Is there a way we can run your tool as part of our CI (to prevent future regressions)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] calvinhkf commented on issue #9442: Inconsistent library
versions notice.
Posted by GitBox <gi...@apache.org>.
calvinhkf commented on issue #9442: Inconsistent library versions notice.
URL: https://github.com/apache/druid/issues/9442#issuecomment-593063553
@ccaominh Thanks for your interests in our tool! This is exactly what we are doing, i.e., to make our research tool either a Maven or Travis plugin to benefit the community. We will let you know when our tool is ready.
BTW, can you help to confirm the detected inconsistencies? Will you fix them? Or you won't potentially because they are intentionally kept due to heavy API incompatibilities? Any feedbacks will be useful for us to improve the tool.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org