You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "mingmwang (via GitHub)" <gi...@apache.org> on 2023/05/08 01:48:35 UTC
[GitHub] [arrow-datafusion] mingmwang opened a new issue, #6277: thread 'tpcds_logical_q64' has overflowed its stack
mingmwang opened a new issue, #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277
### Describe the bug
I run cargo test with latest code base and encounter this error.
### To Reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1554584495
FWIW I also run mac osx and I can't reproduce the problem on main at 412867aa2596a2df93635b6ac31d48aa14111c40
```shell
(arrow_dev) alamb@MacBook-Pro-8:~/Software/arrow-datafusion$ git rev-parse HEAD
412867aa2596a2df93635b6ac31d48aa14111c40
(arrow_dev) alamb@MacBook-Pro-8:~/Software/arrow-datafusion$ RUST_BACKTRACE=1 CARGO_TARGET_DIR=/Users/alamb/Software/target-df cargo test -p datafusion --test tpcds_planning -- tpcds_logical_q64
Finished test [unoptimized + debuginfo] target(s) in 0.27s
Running tests/tpcds_planning.rs (/Users/alamb/Software/target-df/debug/deps/tpcds_planning-66152470902f3ea8)
running 1 test
test tpcds_logical_q64 ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 197 filtered out; finished in 2.04s
(arrow_dev) alamb@MacBook-Pro-8:~/Software/arrow-datafusion$
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mingmwang commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "mingmwang (via GitHub)" <gi...@apache.org>.
mingmwang commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1537658923
The rule was invoked 560 times.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] crepererum commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "crepererum (via GitHub)" <gi...@apache.org>.
crepererum commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1540286512
Uh, Mac OS has only 8k stack size by default? Interesting. I will see what I can do to optimize that. Note though that this may take up to two weeks (traveling, holidays, other work stuff).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mingmwang commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "mingmwang (via GitHub)" <gi...@apache.org>.
mingmwang commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1537643673
@alamb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mingmwang commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "mingmwang (via GitHub)" <gi...@apache.org>.
mingmwang commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1537647162
try_optimize common_subexpr_eliminate.rs:111
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:215
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
optimize_children<datafusion_optimizer::common_subexpr_eliminate::CommonSubexprEliminate> utils.rs:53
try_optimize common_subexpr_eliminate.rs:345
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:316
rewrite_expr common_subexpr_eliminate.rs:99
try_optimize common_subexpr_eliminate.rs:131
optimize_recursively optimizer.rs:400
optimize<datafusion::execution::context::{impl#8}::optimize::{closure_env#3}> optimizer.rs:279
optimize context.rs:1826
{async_fn#0} tpcds_planning.rs:1075
{async_fn#0} tpcds_planning.rs:1045
{async_block#0} tpcds_planning.rs:357
poll<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>> future.rs:125
poll<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>>> future.rs:125
{closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>>>> current_thread.rs:541
[Inlined] tokio::runtime::coop::with_budget coop.rs:102
[Inlined] tokio::runtime::coop::budget coop.rs:68
{closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>>>> current_thread.rs:541
enter<core::task::poll::Poll<core::result::Result<(), datafusion_common::error::DataFusionError>>, tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>>>>> current_thread.rs:350
{closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>>>> current_thread.rs:540
{closure#0}<tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>>>>, core::option::Option<core::result::Result<(), datafusion_common::error::DataFusionError>>> current_thread.rs:615
set<tokio::runtime::scheduler::current_thread::Context, tokio::runtime::scheduler::current_thread::{impl#8}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>>>>, core::option::Option<core::result::Result<(), datafusion_common::error::DataFusionError>>>, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<core::result::Result<(), datafusion_common::error::DataFusionError>>)> scoped_tls.rs:61
enter<tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>>>>, core::option::Option<core::result::Result<(), datafusion_common::error::DataFusionError>>> current_thread.rs:615
block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>>>> current_thread.rs:530
block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>>> current_thread.rs:154
block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), datafusion_common::error::DataFusionError>>>> runtime.rs:282
tpcds_logical_q64 tpcds_planning.rs:357
{closure#0} tpcds_planning.rs:356
call_once<tpcds_planning::tpcds_logical_q64::{closure_env#0}, ()> function.rs:250
[Inlined] core::ops::function::FnOnce::call_once function.rs:250
__rust_begin_short_backtrace<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>> lib.rs:656
[Inlined] _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once boxed.rs:1988
[Inlined] _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once unwind_safe.rs:271
[Inlined] std::panicking::try::do_call panicking.rs:483
[Inlined] std::panicking::try panicking.rs:447
[Inlined] std::panic::catch_unwind panic.rs:140
[Inlined] test::run_test_in_process lib.rs:679
{closure#0} lib.rs:573
[Inlined] test::run_test::run_test_inner::_$u7b$$u7b$closure$u7d$$u7d$ lib.rs:600
__rust_begin_short_backtrace<test::run_test::run_test_inner::{closure_env#1}, ()> backtrace.rs:121
[Inlined] std::thread::Builder::spawn_unchecked_::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$ mod.rs:558
[Inlined] _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once unwind_safe.rs:271
[Inlined] std::panicking::try::do_call panicking.rs:483
[Inlined] std::panicking::try panicking.rs:447
[Inlined] std::panic::catch_unwind panic.rs:140
[Inlined] std::thread::Builder::spawn_unchecked_::_$u7b$$u7b$closure$u7d$$u7d$ mod.rs:557
call_once<std::thread::{impl#0}::spawn_unchecked_::{closure_env#1}<test::run_test::run_test_inner::{closure_env#1}, ()>, ()> function.rs:250
[Inlined] _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once boxed.rs:1988
[Inlined] _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once boxed.rs:1988
thread_start thread.rs:108
_pthread_start 0x000000018fc9026c
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mingmwang commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "mingmwang (via GitHub)" <gi...@apache.org>.
mingmwang commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1537643573
Should be caused by this PR:
https://github.com/apache/arrow-datafusion/pull/6129
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mingmwang commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "mingmwang (via GitHub)" <gi...@apache.org>.
mingmwang commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1537631222
@crepererum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1538561034
This probably only happens in a debug build , but we should fix it if possible
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mingmwang commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "mingmwang (via GitHub)" <gi...@apache.org>.
mingmwang commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1539287170
> 1. cargo test -p datafusion tpcds_planning -- tpcds_logical_q64
> @mingmwang I have some questions:
>
> 1. can you reproduce this with `cargo test -p datafusion tpcds_planning -- tpcds_logical_q64`
Yes, I can reproduce this.
> 2. what is `rustc --version --verbose`
mingmwang@KM27FYXH3G arrow-datafusion % rustc --version
rustc 1.68.0 (2c8cc3432 2023-03-06)
> 3. what is `ulimit -a`
mingmwang@KM27FYXH3G arrow-datafusion % ulimit -a
-t: cpu time (seconds) unlimited
-f: file size (blocks) unlimited
-d: data seg size (kbytes) unlimited
-s: stack size (kbytes) 8176
-c: core file size (blocks) 0
-v: address space (kbytes) unlimited
-l: locked-in-memory size (kbytes) unlimited
-u: processes 10666
-n: file descriptors 256
> 4. what is your OS + version?
macOS Monterey Version 12.5.1 (21G83)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mingmwang commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "mingmwang (via GitHub)" <gi...@apache.org>.
mingmwang commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1537630356
Looks like it is caused by recent changes to rule `CommonSubexprEliminate`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mingmwang closed issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "mingmwang (via GitHub)" <gi...@apache.org>.
mingmwang closed issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
URL: https://github.com/apache/arrow-datafusion/issues/6277
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mingmwang commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "mingmwang (via GitHub)" <gi...@apache.org>.
mingmwang commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1560383452
I can not reproduce the issue now. Just close this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] crepererum commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "crepererum (via GitHub)" <gi...@apache.org>.
crepererum commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1539111385
@mingmwang I have some questions:
1. can you reproduce this with `cargo test -p datafusion tpcds_planning -- tpcds_logical_q64`
2. what is `rustc --version --verbose`
3. what is `ulimit -a`
4. what is your OS + version?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] crepererum commented on issue #6277: thread 'tpcds_logical_q64' has overflowed its stack
Posted by "crepererum (via GitHub)" <gi...@apache.org>.
crepererum commented on issue #6277:
URL: https://github.com/apache/arrow-datafusion/issues/6277#issuecomment-1539093828
I'll have a look.
In general this is an issue w/ many nested data structures and algorithms in DF though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org