[GitHub] [arrow] westonpace commented on issue #36389: [Python][Parquet] pq.write_to_dataset crash from numpy

["westonpace (via GitHub)" <gi...@apache.org>]

westonpace commented on issue #36389:
URL: https://github.com/apache/arrow/issues/36389#issuecomment-1614872958

   I believe it's crashing on exit because the scanner is still running after the `write_to_dataset` call.  This has been a long-standing bug that I haven't been able to find time to work on.
   
   It only affects numpy because normally the scanner is just running some destructors and cleaning up its objects and that is harmless.  However, when the buffers are sourced from numpy, they need to obtain the gil as part of the buffer destruction.  This attempts to obtain the GIL after python has already begun finalizing which causes a crash.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org