[C++] Segfault on RecordBatchWriter::Close (Possible miscompilation?) on OS X 10.15.7

[Carlos Scheidegger <cs...@cscheid.net>]

Hello,

I'm attempting to track down a segfault I'm experiencing on Apache Arrow's C++ libraries. I'm attaching a self-contained, minimal repro (also available in this github gist: https://gist.github.com/cscheid/0f56e1aefe47a4cc04f99836ba8b0bbb).

I'm compiling Apache Arrow's C++ libraries from Github's master branch, using commit 924449eba36acda22ccb319e8de8921c090a4cd2 (HEAD as of 2012-02-21).

On OS X 10.15.7, (Apple clang version 12.0.0 (clang-1200.0.32.21)), the binary segfaults upon calling writer->Close() (line 56). On Ubuntu 20.04, (g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0), the execution succeeds. 

This behavior is consistent (segfault on Apple's clang 12, success on g++ 9.3.0),  regardless of whether I compile the test file with debug or release options, always with -Wall -Werror -Wpedantic. No errors or warnings are reported.

I still think I'm likely doing something wrong, but what makes me suspect a miscompilation is that if I (slightly) instrument Apache Arrow by adding a print statement to cpp/src/arrow/ipc/writer.cc:1006, (simply a 'cerr << "Calling WriteTable" << endl;' statement and the appropriate include directive), I get different behaviors on OS X and Linux.

On Linux, (as expected) I get:

$ ./test_arrow_write
Calling WriteTable
$

On OS X, though, I get:

$ ./test_arrow_write
Calling WriteTable
Calling WriteTable
[1]    3023 segmentation fault  ./test_arrow_write
$

Note the unexpected repeated call to WriteTable. On OS X, I can confirm with a step-through debugger on XCode that the call to writer->Close() somehow ends up in WriteTable(). 

This is where I got stuck, and I'm no longer sure how to proceed. I'll be happy to add more information in case this helps you. (For example, I tried installing clang on Ubuntu 20.04 to check against a third compiler but couldn't get Arrow's CMake to be happy about the configuration process. If someone can walk me through that bit I can try it.)

Thank you very much,
-carlos

Re: [C++] Segfault on RecordBatchWriter::Close (Possible miscompilation?) on OS X 10.15.7

[Carlos Scheidegger <cs...@cscheid.net>]

Folks,

I apologize for the noise - I tracked down the (unrelated) issue. In case other folks run into the same issue: CMake generated compilation commands which included -I/usr/local/include (because of a _different_ dependency on that particular project). That directive prioritized an old version of Arrow on /usr/local/include, and not the one I was actually linking against.

I'm sorry once again.
-carlos

On Sun, Feb 21, 2021, at 4:18 PM, Carlos Scheidegger wrote:
> Hello,
> 
> I'm attempting to track down a segfault I'm experiencing on Apache 
> Arrow's C++ libraries. I'm attaching a self-contained, minimal repro 
> (also available in this github gist: 
> https://gist.github.com/cscheid/0f56e1aefe47a4cc04f99836ba8b0bbb).
> 
> I'm compiling Apache Arrow's C++ libraries from Github's master branch, 
> using commit 924449eba36acda22ccb319e8de8921c090a4cd2 (HEAD as of 
> 2012-02-21).
> 
> On OS X 10.15.7, (Apple clang version 12.0.0 (clang-1200.0.32.21)), the 
> binary segfaults upon calling writer->Close() (line 56). On Ubuntu 
> 20.04, (g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0), the execution 
> succeeds. 
> 
> This behavior is consistent (segfault on Apple's clang 12, success on 
> g++ 9.3.0),  regardless of whether I compile the test file with debug 
> or release options, always with -Wall -Werror -Wpedantic. No errors or 
> warnings are reported.
> 
> I still think I'm likely doing something wrong, but what makes me 
> suspect a miscompilation is that if I (slightly) instrument Apache 
> Arrow by adding a print statement to cpp/src/arrow/ipc/writer.cc:1006, 
> (simply a 'cerr << "Calling WriteTable" << endl;' statement and the 
> appropriate include directive), I get different behaviors on OS X and 
> Linux.
> 
> On Linux, (as expected) I get:
> 
> $ ./test_arrow_write
> Calling WriteTable
> $
> 
> On OS X, though, I get:
> 
> $ ./test_arrow_write
> Calling WriteTable
> Calling WriteTable
> [1]    3023 segmentation fault  ./test_arrow_write
> $
> 
> Note the unexpected repeated call to WriteTable. On OS X, I can confirm 
> with a step-through debugger on XCode that the call to writer->Close() 
> somehow ends up in WriteTable(). 
> 
> This is where I got stuck, and I'm no longer sure how to proceed. I'll 
> be happy to add more information in case this helps you. (For example, 
> I tried installing clang on Ubuntu 20.04 to check against a third 
> compiler but couldn't get Arrow's CMake to be happy about the 
> configuration process. If someone can walk me through that bit I can 
> try it.)
> 
> Thank you very much,
> -carlos
> Attachments:
> * test_arrow_write.cc