[jira] [Commented] (ARROW-17879) [R] Intermittent memory leaks in the valgrind nightly test

["Dewey Dunnington (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17654093#comment-17654093 ] 

Dewey Dunnington commented on ARROW-17879:
------------------------------------------

For a long time the job was just timing out, but we had a recent pass that shows that this is still a problem:

https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=42194&view=logs&j=0da5d1d9-276d-5173-c4c4-9d4d4ed14fdb&t=d9b15392-e4ce-5e4c-0c8c-b69645229181&l=27753

The job finishing was pretty close to the 6 hour timeout, so I wonder if we're seeing the leaks because Azure is trying to terminate the process or something. I'd like to try to replicate this locally first!

{noformat}
==7045== HEAP SUMMARY:
==7045==     in use at exit: 311,868,848 bytes in 65,303 blocks
==7045==   total heap usage: 13,472,680 allocs, 13,407,377 frees, 10,839,858,195 bytes allocated
==7045== 
==7045==    by 0x11219CC1: arrow::compute::MakeExecNode(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, arrow::compute::ExecPlan*, std::vector<arrow::compute::ExecNode*, std::allocator<arrow::compute::ExecNode*> >, arrow::compute::ExecNodeOptions const&, arrow::compute::ExecFactoryRegistry*) (exec_plan.h:342)
==7045==    by 0x11212924: MakeExecNodeOrStop(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, arrow::compute::ExecPlan*, std::vector<arrow::compute::ExecNode*, std::allocator<arrow::compute::ExecNode*> >, arrow::compute::ExecNodeOptions const&) (compute-exec.cpp:68)
==7045==    by 0x1121479C: ExecNode_Filter(std::shared_ptr<arrow::compute::ExecNode> const&, std::shared_ptr<arrow::compute::Expression> const&) (compute-exec.cpp:373)
==7045==    by 0x11195B1B: _arrow_ExecNode_Filter (arrowExports.cpp:1027)
==7045==    by 0x4956E74: R_doDotCall (dotcode.c:871)
==7045== 
==7045== LEAK SUMMARY:
==7045==    definitely lost: 592 bytes in 5 blocks
==7045==    indirectly lost: 119,033 bytes in 1,238 blocks
==7045==      possibly lost: 39,057 bytes in 428 blocks
==7045==    still reachable: 311,710,166 bytes in 63,632 blocks
==7045==                       of which reachable via heuristic:
==7045==                         newarray           : 4,264 bytes in 1 blocks
==7045==                         multipleinheritance: 6,064 bytes in 117 blocks
==7045==         suppressed: 0 bytes in 0 blocks
==7045== Reachable blocks (those to which a pointer was found) are not shown.
==7045== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==7045== 
==7045== For lists of detected and suppressed errors, rerun with: -s
==7045== ERROR SUMMARY: 206 errors from 206 contexts (suppressed: 0 from 0)

{noformat}


> [R] Intermittent memory leaks in the valgrind nightly test
> ----------------------------------------------------------
>
>                 Key: ARROW-17879
>                 URL: https://issues.apache.org/jira/browse/ARROW-17879
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: R
>            Reporter: Dewey Dunnington
>            Assignee: Dewey Dunnington
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 11.0.0
>
>          Time Spent: 6h 20m
>  Remaining Estimate: 0h
>
> The memory leaks that were fixed by a workaround before the last release (ARROW-17252) are present again. I had hoped that the improvements to the captured R thread infrastructure in ARROW-11841 and ARROW-17178 would fix this; however, they don't (and it's not even clear that the failures are related to that, since as part of diagnosing those failures the last time I disabled the safe call infrastructure completely and was still able to observe failures).
> These failures need to be debugged before the release!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)