You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Wei Yi Luo <we...@est.tech> on 2022/07/11 05:03:08 UTC
Questions regarding boost version of native client
Hi all,
When trying to use a recent boost 1.78.0 library (dynamically loaded by application process), core file was triggered by native client library during pool creation.
Her comes the questions:
1. Is it recommended to tweak boost version by application and how risky it could be?
2.
3. Any plan when to uplift boost version towards 1.78.0 or later?
Thanks!
Br
Bo
Re: Questions regarding boost version of native client
Posted by Wei Yi Luo <we...@est.tech>.
Hi Jacob,
I got the details from the application guy as following:
The Geode client library is built in a SLES docker:
compiler:
g++ (SUSE Linux) 7.3.1 20180323 [gcc-7-branch revision 258812]
with following options:
cmake -DCMAKE_BUILD_TYPE=RELWITHDEBINFO -DWITH_IPV6=ON -DCMAKE_INSTALL_PREFIX=$GEODE_NATIVE_DIR
The application boost is built based in another SLES docker:
compiler:
g++ (SUSE Linux) 7.5.0
The application boost library, with that compiler, is built with the following compiler flags:
-g -O2 -D_FORTIFY_SOURCE=2 -D_GLIBCXX_ASSERTIONS -fstack-protector-strong -fasynchronous-unwind-tables -fexceptions -fpic -Wformat -Werror=format-security
And with the following boost build system options:
./b2 a -d2 "-prefix=${PREPARE_DIR}/boost" "-j$(($(nproc) - 1))" --layout=system --without-python --without-mpi --without-graph_parallel variant=release address-model=64 link=shared threading=multi debug-symbols=on install
Thanks!
Br
Bo
________________________________
From: Jacob Barrett <ja...@vmware.com>
Sent: Friday, July 22, 2022 1:12 AM
To: user@geode.apache.org <us...@geode.apache.org>
Subject: Re: Questions regarding boost version of native client
There shouldn’t be any Boost symbols exported by the Geode client library to conflict with applications using Boost. Providing the linkage errors might help in isolating the issue. It is possible that something got buggered up and some of the internal symbols are being exported. What OS and compiler are you using?
On Jul 20, 2022, at 11:20 PM, Wei Yi Luo <we...@est.tech>> wrote:
⚠ External Email
Hi Mario,
Thanks for the advices.
I got the feedback from the application side:
The problem is that geode-library uses boost system linked statically, and any application that uses boost system loaded dynamically will fail, because it seems boost system in 1.76 (or earlier) and 1.78 are not compatible. For previous boost versions that was not a problem, but since 1.78 it seems so.
Actually, they tried to rebuild the library with 1.78 and it works fine, that's why I asked the question: how risky it could be in case the application modify the boost version and re-build the geode-lib using 1.78?
When it comes to the PR of uplift to 1.79, how soon would it be then?
Or do you think such a Jira case could be valid as another approach: Is it possible not to link boost statically in geode native library, or at least, give the user application the chance not to do it?
Br
Bo
________________________________
From: Mario Salazar de Torres <ma...@est.tech>>
Sent: Monday, July 11, 2022 7:28 PM
To: user@geode.apache.org<ma...@geode.apache.org> <us...@geode.apache.org>>
Subject: Re: Questions regarding boost version of native client
Hi,
I've been reviewing geode-native headers and there is no boost code exposed there.
Also, as can be observed here: https://github.com/apache/geode-native/blob/26fe7275cce96b5a3ea785e38f340c9c9c6773d5/dependencies/boost/CMakeLists.txt#L35<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode-native%2Fblob%2F26fe7275cce96b5a3ea785e38f340c9c9c6773d5%2Fdependencies%2Fboost%2FCMakeLists.txt%23L35&data=05%7C01%7Cjabarrett%40vmware.com%7C9c4e98c9ee6c4917384208da6ae12914%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637939812562221544%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=m5xFNPLPTTg2x10tDMD1iQVkIdQ6a6TVRzP4mKf4ebY%3D&reserved=0>, Boost is statically linked to geode-library.
[https://opengraph.githubassets.com/3bd989ea7a80472c63a535297986dbd6833608a54b7fde59d546b21e45d10ec2/apache/geode-native]<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode-native%2Fblob%2F26fe7275cce96b5a3ea785e38f340c9c9c6773d5%2Fdependencies%2Fboost%2FCMakeLists.txt%23L35&data=05%7C01%7Cjabarrett%40vmware.com%7C9c4e98c9ee6c4917384208da6ae12914%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637939812562221544%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=m5xFNPLPTTg2x10tDMD1iQVkIdQ6a6TVRzP4mKf4ebY%3D&reserved=0>
geode-native/CMakeLists.txt at 26fe7275cce96b5a3ea785e38f340c9c9c6773d5 · apache/geode-native<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode-native%2Fblob%2F26fe7275cce96b5a3ea785e38f340c9c9c6773d5%2Fdependencies%2Fboost%2FCMakeLists.txt%23L35&data=05%7C01%7Cjabarrett%40vmware.com%7C9c4e98c9ee6c4917384208da6ae12914%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637939812562221544%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=m5xFNPLPTTg2x10tDMD1iQVkIdQ6a6TVRzP4mKf4ebY%3D&reserved=0>
Apache Geode Native. Contribute to apache/geode-native development by creating an account on GitHub.
github.com<http://github.com/>
So, the only problematic scenario in terms of Boost linking conflicts would be if you are using the same building environment for your application and for Geode.
My guess here is that it could happen that Boost headers from your environment (at 1.78.0 apparently) are used to compile Geode-Native, instead of the ones
downloaded in the dependencies (1.76.0) and I suppose that 1.76.0 and 1.78.0 have API compatible changes, but breaking ones in term of ABI, thus causing the coredump.
Usually, in order to compile Geode Native I use one of the Docker images<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode-native%2Ftree%2Fdevelop%2Fdocker&data=05%7C01%7Cjabarrett%40vmware.com%7C9c4e98c9ee6c4917384208da6ae12914%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637939812562221544%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rsP3v2E%2BPTYHWWawJTR4vxGv0Gk7dPwn4X9Gs0qQr9A%3D&reserved=0> already included in the repository, to avoid dependency issues.
If my guess is still not correct I'd encourage you to open a Jira case, so the issue can be tackled by the community.
Regarding plans to uplift to 1.78.0 or later, there is currently an open PR by Michael Oleske ([http://PR<http://pr/> #967]PR #967) to uplift to 1.79, it just requires some care to Windows plat, so I guess it shouldn't take long...
BR
/Mario
________________________________
From: Wei Yi Luo <we...@est.tech>>
Sent: Monday, July 11, 2022 7:03 AM
To: user@geode.apache.org<ma...@geode.apache.org> <us...@geode.apache.org>>
Cc: dev@geode.apache.org<ma...@geode.apache.org> <de...@geode.apache.org>>
Subject: Questions regarding boost version of native client
Hi all,
When trying to use a recent boost 1.78.0 library (dynamically loaded by application process), core file was triggered by native client library during pool creation.
Her comes the questions:
1. Is it recommended to tweak boost version by application and how risky it could be?
2.
3. Any plan when to uplift boost version towards 1.78.0 or later?
Thanks!
Br
Bo
________________________________
⚠ External Email: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender.
Re: Questions regarding boost version of native client
Posted by Jacob Barrett <ja...@vmware.com>.
There shouldn’t be any Boost symbols exported by the Geode client library to conflict with applications using Boost. Providing the linkage errors might help in isolating the issue. It is possible that something got buggered up and some of the internal symbols are being exported. What OS and compiler are you using?
On Jul 20, 2022, at 11:20 PM, Wei Yi Luo <we...@est.tech>> wrote:
⚠ External Email
Hi Mario,
Thanks for the advices.
I got the feedback from the application side:
The problem is that geode-library uses boost system linked statically, and any application that uses boost system loaded dynamically will fail, because it seems boost system in 1.76 (or earlier) and 1.78 are not compatible. For previous boost versions that was not a problem, but since 1.78 it seems so.
Actually, they tried to rebuild the library with 1.78 and it works fine, that's why I asked the question: how risky it could be in case the application modify the boost version and re-build the geode-lib using 1.78?
When it comes to the PR of uplift to 1.79, how soon would it be then?
Or do you think such a Jira case could be valid as another approach: Is it possible not to link boost statically in geode native library, or at least, give the user application the chance not to do it?
Br
Bo
________________________________
From: Mario Salazar de Torres <ma...@est.tech>>
Sent: Monday, July 11, 2022 7:28 PM
To: user@geode.apache.org<ma...@geode.apache.org> <us...@geode.apache.org>>
Subject: Re: Questions regarding boost version of native client
Hi,
I've been reviewing geode-native headers and there is no boost code exposed there.
Also, as can be observed here: https://github.com/apache/geode-native/blob/26fe7275cce96b5a3ea785e38f340c9c9c6773d5/dependencies/boost/CMakeLists.txt#L35<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode-native%2Fblob%2F26fe7275cce96b5a3ea785e38f340c9c9c6773d5%2Fdependencies%2Fboost%2FCMakeLists.txt%23L35&data=05%7C01%7Cjabarrett%40vmware.com%7C9c4e98c9ee6c4917384208da6ae12914%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637939812562221544%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=m5xFNPLPTTg2x10tDMD1iQVkIdQ6a6TVRzP4mKf4ebY%3D&reserved=0>, Boost is statically linked to geode-library.
[https://opengraph.githubassets.com/3bd989ea7a80472c63a535297986dbd6833608a54b7fde59d546b21e45d10ec2/apache/geode-native]<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode-native%2Fblob%2F26fe7275cce96b5a3ea785e38f340c9c9c6773d5%2Fdependencies%2Fboost%2FCMakeLists.txt%23L35&data=05%7C01%7Cjabarrett%40vmware.com%7C9c4e98c9ee6c4917384208da6ae12914%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637939812562221544%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=m5xFNPLPTTg2x10tDMD1iQVkIdQ6a6TVRzP4mKf4ebY%3D&reserved=0>
geode-native/CMakeLists.txt at 26fe7275cce96b5a3ea785e38f340c9c9c6773d5 · apache/geode-native<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode-native%2Fblob%2F26fe7275cce96b5a3ea785e38f340c9c9c6773d5%2Fdependencies%2Fboost%2FCMakeLists.txt%23L35&data=05%7C01%7Cjabarrett%40vmware.com%7C9c4e98c9ee6c4917384208da6ae12914%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637939812562221544%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=m5xFNPLPTTg2x10tDMD1iQVkIdQ6a6TVRzP4mKf4ebY%3D&reserved=0>
Apache Geode Native. Contribute to apache/geode-native development by creating an account on GitHub.
github.com<http://github.com/>
So, the only problematic scenario in terms of Boost linking conflicts would be if you are using the same building environment for your application and for Geode.
My guess here is that it could happen that Boost headers from your environment (at 1.78.0 apparently) are used to compile Geode-Native, instead of the ones
downloaded in the dependencies (1.76.0) and I suppose that 1.76.0 and 1.78.0 have API compatible changes, but breaking ones in term of ABI, thus causing the coredump.
Usually, in order to compile Geode Native I use one of the Docker images<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode-native%2Ftree%2Fdevelop%2Fdocker&data=05%7C01%7Cjabarrett%40vmware.com%7C9c4e98c9ee6c4917384208da6ae12914%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637939812562221544%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rsP3v2E%2BPTYHWWawJTR4vxGv0Gk7dPwn4X9Gs0qQr9A%3D&reserved=0> already included in the repository, to avoid dependency issues.
If my guess is still not correct I'd encourage you to open a Jira case, so the issue can be tackled by the community.
Regarding plans to uplift to 1.78.0 or later, there is currently an open PR by Michael Oleske ([http://PR<http://pr/> #967]PR #967) to uplift to 1.79, it just requires some care to Windows plat, so I guess it shouldn't take long...
BR
/Mario
________________________________
From: Wei Yi Luo <we...@est.tech>>
Sent: Monday, July 11, 2022 7:03 AM
To: user@geode.apache.org<ma...@geode.apache.org> <us...@geode.apache.org>>
Cc: dev@geode.apache.org<ma...@geode.apache.org> <de...@geode.apache.org>>
Subject: Questions regarding boost version of native client
Hi all,
When trying to use a recent boost 1.78.0 library (dynamically loaded by application process), core file was triggered by native client library during pool creation.
Her comes the questions:
1. Is it recommended to tweak boost version by application and how risky it could be?
2.
3. Any plan when to uplift boost version towards 1.78.0 or later?
Thanks!
Br
Bo
________________________________
⚠ External Email: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender.
Re: Questions regarding boost version of native client
Posted by Wei Yi Luo <we...@est.tech>.
Hi Mario,
Thanks for the advices.
I got the feedback from the application side:
The problem is that geode-library uses boost system linked statically, and any application that uses boost system loaded dynamically will fail, because it seems boost system in 1.76 (or earlier) and 1.78 are not compatible. For previous boost versions that was not a problem, but since 1.78 it seems so.
Actually, they tried to rebuild the library with 1.78 and it works fine, that's why I asked the question: how risky it could be in case the application modify the boost version and re-build the geode-lib using 1.78?
When it comes to the PR of uplift to 1.79, how soon would it be then?
Or do you think such a Jira case could be valid as another approach: Is it possible not to link boost statically in geode native library, or at least, give the user application the chance not to do it?
Br
Bo
________________________________
From: Mario Salazar de Torres <ma...@est.tech>
Sent: Monday, July 11, 2022 7:28 PM
To: user@geode.apache.org <us...@geode.apache.org>
Subject: Re: Questions regarding boost version of native client
Hi,
I've been reviewing geode-native headers and there is no boost code exposed there.
Also, as can be observed here: https://github.com/apache/geode-native/blob/26fe7275cce96b5a3ea785e38f340c9c9c6773d5/dependencies/boost/CMakeLists.txt#L35, Boost is statically linked to geode-library.
[https://opengraph.githubassets.com/3bd989ea7a80472c63a535297986dbd6833608a54b7fde59d546b21e45d10ec2/apache/geode-native]<https://github.com/apache/geode-native/blob/26fe7275cce96b5a3ea785e38f340c9c9c6773d5/dependencies/boost/CMakeLists.txt#L35>
geode-native/CMakeLists.txt at 26fe7275cce96b5a3ea785e38f340c9c9c6773d5 · apache/geode-native<https://github.com/apache/geode-native/blob/26fe7275cce96b5a3ea785e38f340c9c9c6773d5/dependencies/boost/CMakeLists.txt#L35>
Apache Geode Native. Contribute to apache/geode-native development by creating an account on GitHub.
github.com
So, the only problematic scenario in terms of Boost linking conflicts would be if you are using the same building environment for your application and for Geode.
My guess here is that it could happen that Boost headers from your environment (at 1.78.0 apparently) are used to compile Geode-Native, instead of the ones
downloaded in the dependencies (1.76.0) and I suppose that 1.76.0 and 1.78.0 have API compatible changes, but breaking ones in term of ABI, thus causing the coredump.
Usually, in order to compile Geode Native I use one of the Docker images<https://github.com/apache/geode-native/tree/develop/docker> already included in the repository, to avoid dependency issues.
If my guess is still not correct I'd encourage you to open a Jira case, so the issue can be tackled by the community.
Regarding plans to uplift to 1.78.0 or later, there is currently an open PR by Michael Oleske ([http://PR #967]PR #967) to uplift to 1.79, it just requires some care to Windows plat, so I guess it shouldn't take long...
BR
/Mario
________________________________
From: Wei Yi Luo <we...@est.tech>
Sent: Monday, July 11, 2022 7:03 AM
To: user@geode.apache.org <us...@geode.apache.org>
Cc: dev@geode.apache.org <de...@geode.apache.org>
Subject: Questions regarding boost version of native client
Hi all,
When trying to use a recent boost 1.78.0 library (dynamically loaded by application process), core file was triggered by native client library during pool creation.
Her comes the questions:
1. Is it recommended to tweak boost version by application and how risky it could be?
2.
3. Any plan when to uplift boost version towards 1.78.0 or later?
Thanks!
Br
Bo
Re: Questions regarding boost version of native client
Posted by Mario Salazar de Torres <ma...@est.tech>.
Hi,
I've been reviewing geode-native headers and there is no boost code exposed there.
Also, as can be observed here: https://github.com/apache/geode-native/blob/26fe7275cce96b5a3ea785e38f340c9c9c6773d5/dependencies/boost/CMakeLists.txt#L35, Boost is statically linked to geode-library.
[https://opengraph.githubassets.com/3bd989ea7a80472c63a535297986dbd6833608a54b7fde59d546b21e45d10ec2/apache/geode-native]<https://github.com/apache/geode-native/blob/26fe7275cce96b5a3ea785e38f340c9c9c6773d5/dependencies/boost/CMakeLists.txt#L35>
geode-native/CMakeLists.txt at 26fe7275cce96b5a3ea785e38f340c9c9c6773d5 · apache/geode-native<https://github.com/apache/geode-native/blob/26fe7275cce96b5a3ea785e38f340c9c9c6773d5/dependencies/boost/CMakeLists.txt#L35>
Apache Geode Native. Contribute to apache/geode-native development by creating an account on GitHub.
github.com
So, the only problematic scenario in terms of Boost linking conflicts would be if you are using the same building environment for your application and for Geode.
My guess here is that it could happen that Boost headers from your environment (at 1.78.0 apparently) are used to compile Geode-Native, instead of the ones
downloaded in the dependencies (1.76.0) and I suppose that 1.76.0 and 1.78.0 have API compatible changes, but breaking ones in term of ABI, thus causing the coredump.
Usually, in order to compile Geode Native I use one of the Docker images<https://github.com/apache/geode-native/tree/develop/docker> already included in the repository, to avoid dependency issues.
If my guess is still not correct I'd encourage you to open a Jira case, so the issue can be tackled by the community.
Regarding plans to uplift to 1.78.0 or later, there is currently an open PR by Michael Oleske (PR #967<http://PR%20#967>) to uplift to 1.79, it just requires some care to Windows plat, so I guess it shouldn't take long...
BR
/Mario
________________________________
From: Wei Yi Luo <we...@est.tech>
Sent: Monday, July 11, 2022 7:03 AM
To: user@geode.apache.org <us...@geode.apache.org>
Cc: dev@geode.apache.org <de...@geode.apache.org>
Subject: Questions regarding boost version of native client
Hi all,
When trying to use a recent boost 1.78.0 library (dynamically loaded by application process), core file was triggered by native client library during pool creation.
Her comes the questions:
1. Is it recommended to tweak boost version by application and how risky it could be?
2.
3. Any plan when to uplift boost version towards 1.78.0 or later?
Thanks!
Br
Bo