You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/01/27 10:38:21 UTC
[GitHub] [incubator-doris] xinyiZzz opened a new pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
xinyiZzz opened a new pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911
## Problem Summary:
Error details in be.out:
````
/home/disk3/zxy/baidu/bdg/doris/core/output/be/lib/palo_be: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /home/disk3/zxy/ baidu/bdg/doris/core/output/be/lib/palo_be)
````
nm looks at palo_be and finds that the method that depends on GLIBC_2.18 is `__cxa_thread_atexit_impl`
I found that the `__cxa_thread_atexit_impl` method was introduced by glibc to support thread_local variables
https://sourceware.org/glibc/wiki/Destructor%20support%20for%20thread_local%20variables
It was no problem to compile through gcc10 before, our development machine glibc version only reaches GLIBC_2.12, and it is not allowed to upgrade the glibc version...
So, how can I avoid this problem, please help
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
morningman commented on pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911#issuecomment-1023372778
> Is it for some future PRs with thread local?
Yes, @xinyiZzz is developing memory tracking and management based on thread local.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] amosbird edited a comment on pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
amosbird edited a comment on pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911#issuecomment-1023508528
I did some investigation. It's not easy to mask this symbol, which is meant to be added to the libc++abi. It's also not safe to use the polyfill implementation from some libc++api directly. I'd suggest the following:
1. use clang toolchain
2. keep thread_local with trivial destructor if gcc is used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] amosbird commented on pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
amosbird commented on pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911#issuecomment-1023369852
Is it for some future PRs with thread local?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz closed pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
xinyiZzz closed pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] amosbird commented on pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
amosbird commented on pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911#issuecomment-1023375876
> > Is it for some future PRs with thread local?
>
> Yes, @xinyiZzz is developing memory tracking and management based on thread local. And this is just a PR for troubleshooting.
Cool. Will take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz commented on pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
xinyiZzz commented on pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911#issuecomment-1023917862
> > Regarding the second point, do you mean changing the thread_local variable to a member variable of the class? For example, abstract a parent class to save this variable. or other meanings.
>
> You can redesign the implementation of your thread_local structs to have trivial destructor. The definition is in https://en.cppreference.com/w/cpp/language/destructor .
I think I understand trivial destructor,
For non-trivial classes in thread_local, such as std::string, I plan to store them as pointers.
To ensure that thread_local is trivial, these non-trivial pointers will uniformly call destructors elsewhere.
thanks again, very helpful @amosbird
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz commented on pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
xinyiZzz commented on pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911#issuecomment-1023861894
> I did some investigation. It's not easy to mask this symbol, which is meant to be added to the libc++abi. It's also not safe to use the polyfill implementation from some libc++api directly. I'd suggest the following:
>
> 1. use clang toolchain
> 2. keep thread_local with trivial destructor if gcc is used.
Thanks for the answer~ @amosbird
Compiling with clang is indeed fine.
Regarding the second point, do you mean changing the thread_local variable to a member variable of the class? For example, abstract a parent class to save this variable. or other meanings.
My pr: https://github.com/apache/incubator-doris/pull/7198
I want to track all memory requests during a thread lifetime via thread_local + TCMalloc hook, like Clickhouse does.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] amosbird commented on pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
amosbird commented on pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911#issuecomment-1023889347
> Regarding the second point, do you mean changing the thread_local variable to a member variable of the class? For example, abstract a parent class to save this variable. or other meanings.
You can redesign the implementation of your thread_local structs to have trivial destructor. The definition is in https://en.cppreference.com/w/cpp/language/destructor .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] amosbird commented on pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
amosbird commented on pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911#issuecomment-1023508528
I did some investigation. It's not easy to mask the symbol, which is from libc++abi https://libcxxabi.llvm.org/spec.html . It's also not safe to use the polyfill implementation from some libc++api directly. I'd suggest the following:
1. use clang toolchain
2. keep thread_local with trivial destructor if gcc is used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman edited a comment on pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
morningman edited a comment on pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911#issuecomment-1023372778
> Is it for some future PRs with thread local?
Yes, @xinyiZzz is developing memory tracking and management based on thread local.
And this is just a PR for troubleshooting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
morningman commented on pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911#issuecomment-1023367102
PTAL @amosbird , if you have time, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz edited a comment on pull request #7911: [Help] gcc11 compiles `thread_local` variable, BE start: version `GLIBC_2.18' not found
Posted by GitBox <gi...@apache.org>.
xinyiZzz edited a comment on pull request #7911:
URL: https://github.com/apache/incubator-doris/pull/7911#issuecomment-1023917862
> > Regarding the second point, do you mean changing the thread_local variable to a member variable of the class? For example, abstract a parent class to save this variable. or other meanings.
>
> You can redesign the implementation of your thread_local structs to have trivial destructor. The definition is in https://en.cppreference.com/w/cpp/language/destructor .
I think I understand trivial destructor,
For non-trivial variables in thread_local, such as std::string, I plan to store them as pointers.
To ensure that thread_local is trivial, these non-trivial pointers will uniformly call destructors elsewhere.
And try to use POD type.
thanks again, very helpful @amosbird
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org