You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@celix.apache.org by "PengZheng (via GitHub)" <gi...@apache.org> on 2023/09/25 02:33:41 UTC

[PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

PengZheng opened a new pull request, #657:
URL: https://github.com/apache/celix/pull/657

   It avoids a testing crash.
   
   Considering we just make bundle uninstall product-ready in 2.4.0 and OSGi permits uninstalling a bundle independent of other bundles, #653 should be addressed in the next release. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

Posted by "pnoltes (via GitHub)" <gi...@apache.org>.
pnoltes commented on PR #657:
URL: https://github.com/apache/celix/pull/657#issuecomment-1734215064

   > Whew! I have fixed these leaks lurking in our tests for a very long time. Note that to debug such issue, linux ASLR should be turned off by `echo 0 | sudo tee /proc/sys/kernel/randomize_va_space` so that we can ask debugger which shared object these leakers reported by ASAN belong to at a appropriate time when the offending shared object is still in address space.
   > 
   > IMHO, we should **warn our users very loudly in our documentation** of these shared object initialization/de-initialization pitfalls. Also, it should be pointed out to our users that the standard place to deal with these is a customized launcher. Last but not the lease, `libffi` is very special in that there is no way to clean its global allocation (correct me if I were wrong), thus it should be kept in address space if possible (by linking it to the executable). Keep loading and unloading libffi will wreak a havoc.
   > 
   > WDYT? @pnoltes @rlenferink
   
   I agree, both in warning for init/deinit library requirements in the documentation and that this should be solved in a custom launcher. 
   
   An alternative to consider is to provided init/deinit libs that use the `__attribute__((constructor))` / `__attribute__((destructor))` to initialize / deinitialize libraries. These library then needs to be linked against the generated launcher. 
   
   But IMO it is clearer to do this in the launcher.  Maybe some effort can be made to make it possible that a custom launcher can easily use the `PROPERTIES` provided in the `add_celix_container` (as is currently done in the generated main.c/main.cc file).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

Posted by "PengZheng (via GitHub)" <gi...@apache.org>.
PengZheng commented on PR #657:
URL: https://github.com/apache/celix/pull/657#issuecomment-1732968718

   Another leak in `run_test_tm_scoped` is caused by `tramp_table_alloc` in libffi.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #657:
URL: https://github.com/apache/celix/pull/657#issuecomment-1732816437

   ## [Codecov](https://app.codecov.io/gh/apache/celix/pull/657?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   > Merging [#657](https://app.codecov.io/gh/apache/celix/pull/657?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (f371369) into [master](https://app.codecov.io/gh/apache/celix/commit/e7aee1259a4c61463be8fcfa5dd4612a3a756192?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (e7aee12) will **increase** coverage by `0.05%`.
   > The diff coverage is `n/a`.
   
   ```diff
   @@            Coverage Diff             @@
   ##           master     #657      +/-   ##
   ==========================================
   + Coverage   81.60%   81.66%   +0.05%     
   ==========================================
     Files         260      260              
     Lines       34675    34675              
   ==========================================
   + Hits        28297    28316      +19     
   + Misses       6378     6359      -19     
   ```
   
   
   | [Files Changed](https://app.codecov.io/gh/apache/celix/pull/657?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [...xx\_remote\_services/admin/src/RemoteServiceAdmin.cc](https://app.codecov.io/gh/apache/celix/pull/657?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YnVuZGxlcy9jeHhfcmVtb3RlX3NlcnZpY2VzL2FkbWluL3NyYy9SZW1vdGVTZXJ2aWNlQWRtaW4uY2M=) | `78.70% <ø> (ø)` | |
   
   ... and [6 files with indirect coverage changes](https://app.codecov.io/gh/apache/celix/pull/657/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   
   :mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

Posted by "PengZheng (via GitHub)" <gi...@apache.org>.
PengZheng merged PR #657:
URL: https://github.com/apache/celix/pull/657


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

Posted by "PengZheng (via GitHub)" <gi...@apache.org>.
PengZheng commented on PR #657:
URL: https://github.com/apache/celix/pull/657#issuecomment-1733245685

   Whew! I have fixed these leaks lurking in our tests for a very long time.
   Note that to debug such issue, linux ASLR should be turned off by `echo 0 | sudo tee /proc/sys/kernel/randomize_va_space` so that we can ask debugger which shared object these leakers reported by ASAN belong to at a appropriate time when the offending shared object is still in address space.
   
   IMHO, we should **warn our users very loudly in our documentation** of these shared object initialization/de-initialization pitfalls.
   Also, it should be pointed out to our users that the standard place to deal with these is a customized launcher.
   Last but not the lease, `libffi` is very special in that there is no way to clean its global allocation (correct me if I were wrong), thus it should be kept in address space if possible (by linking it to the executable). Keep loading and unloading libffi will wreak a havoc. 
   
   WDYT? @pnoltes @rlenferink 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

Posted by "PengZheng (via GitHub)" <gi...@apache.org>.
PengZheng commented on PR #657:
URL: https://github.com/apache/celix/pull/657#issuecomment-1734734401

   > An alternative to consider is to provided init/deinit libs that use the __attribute__((constructor)) / __attribute__((destructor)) to initialize / deinitialize libraries. These library then needs to be linked against the generated launcher.
   
   > But IMO it is clearer to do this in the launcher. Maybe some effort can be made to make it possible that a custom launcher can easily use the PROPERTIES provided in the add_celix_container (as is currently done in the generated main.c/main.cc file).
   
   Indeed, manual initialization in launcher is less than ideal.
   The biggest trouble is these initialization functions have options, e.g. `mg_init_library`. Different bundles may have different requirements, and thus set different options.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

Posted by "PengZheng (via GitHub)" <gi...@apache.org>.
PengZheng commented on PR #657:
URL: https://github.com/apache/celix/pull/657#issuecomment-1732880805

   The last commit enabling RelWithDebInfo manifests additional memory leaks:
   
   ```
   	 25 - run_test_tm_scoped (Failed)
   	 26 - run_test_rsa_dfi (Failed)
   	 41 - pubsub_websocket_v2_tests (Failed)
   ```
   
   I can verify load/unload remote_service_admin_dfi will lead to ASAN report.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

Posted by "PengZheng (via GitHub)" <gi...@apache.org>.
PengZheng commented on PR #657:
URL: https://github.com/apache/celix/pull/657#issuecomment-1733096914

   The failure of `run_test_rsa_dfi` brings libxml2 into attention. 
   As noted in its documentation of `xmlCleanupParser`:
   
   > It's sometimes very hard to guess if libxml2 is in use in the application, some libraries or plugins may use it without notice. In case of doubt abstain from calling this function or do it just before calling exit() to avoid leak reports from valgrind !
   
   https://gnome.pages.gitlab.gnome.org/libxml2/devhelp/libxml2-parser.html#xmlCleanupParser
   And we are a plugin framework. Thus we should remind our user of this pitfall.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

Posted by "PengZheng (via GitHub)" <gi...@apache.org>.
PengZheng commented on PR #657:
URL: https://github.com/apache/celix/pull/657#issuecomment-1732949706

   One leak turns out to be caused by `mg_init_library`:
   
   > This function is new in version 1.9 (as dummy implementation) and effective only from version 1.10. For compatibility reasons, other functions (such as [mg_start();](https://github.com/civetweb/civetweb/blob/master/docs/api/mg_start.md)) will initialize the required features as well, but they will no longer do a de-initialization, leaving a memory leak when the library is unloaded.
   
   https://github.com/civetweb/civetweb/blob/master/docs/api/mg_init_library.md


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

Posted by "PengZheng (via GitHub)" <gi...@apache.org>.
PengZheng commented on PR #657:
URL: https://github.com/apache/celix/pull/657#issuecomment-1734760578

   It's interesting to observe that libxml2 does has its destructor defined:
   
   https://github.com/GNOME/libxml2/blob/c7ff438b830d8ebeac684f3f8ad3229587c88373/threads.c#L665-L677
   ```C
   #if defined(HAVE_ATTRIBUTE_DESTRUCTOR) && !defined(LIBXML_STATIC) && \
       !defined(_WIN32)
   static void
   ATTRIBUTE_DESTRUCTOR
   xmlDestructor(void) {
       /*
        * Calling custom deallocation functions in a destructor can cause
        * problems, for example with Nokogiri.
        */
       if (xmlFree == free)
           xmlCleanupParser();
   }
   #endif
   ```
   
   Thus there is no need to call `xmlCleanupParser` in our application code, unless we are using static libxml2 or custom deallocation function.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Workaround the lifecycle management issue of IImportRegistration. (celix)

Posted by "pnoltes (via GitHub)" <gi...@apache.org>.
pnoltes commented on PR #657:
URL: https://github.com/apache/celix/pull/657#issuecomment-1734216296

   IMO this PR is also needed for the 2.4.0 release, so I will wait for a merge before starting a new vote. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org