You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "L3o-pold (via GitHub)" <gi...@apache.org> on 2026/07/10 22:06:56 UTC
[I] [C++][Python] SIGSEGV in bundled mimalloc mi_thread_init when libarrow is first loaded on a non-main thread that exits (mimalloc 3.3.x, pyarrow 25.0.0) [arrow]
L3o-pold opened a new issue, #50471:
URL: https://github.com/apache/arrow/issues/50471
### Describe the bug, including details regarding any error messages, version, and platform
pyarrow 25.0.0 segfaults deterministically when `pyarrow` is first imported on a **non-main thread that later exits**, and a different thread then performs its first Arrow allocation. The crash is inside the bundled mimalloc's thread initialization:
```
Exception Type: EXC_BAD_ACCESS (SIGSEGV) KERN_INVALID_ADDRESS at 0x0000000000000018
libarrow.2500.dylib mi_heap_main
libarrow.2500.dylib mi_thread_init
libarrow.2500.dylib _mi_malloc_generic
libarrow.2500.dylib mi_theap_malloc_zero_aligned_at_generic
libarrow.2500.dylib arrow::(anonymous namespace)::MimallocAllocator::AllocateAligned(long long, long long, unsigned char**)
libarrow.2500.dylib arrow::BaseMemoryPoolImpl<arrow::(anonymous namespace)::MimallocAllocator>::Allocate(...)
libarrow.2500.dylib arrow::PoolBuffer::Reserve(long long)
libarrow.2500.dylib arrow::PoolBuffer::Resize(long long, bool)
libarrow_python.2500.dylib arrow::py::ConvertPySequence / arrow::py::NdarrayToArrow
```
#### Minimal reproduction
```python
import threading
def imp():
import pyarrow # dlopens libarrow -> bundled mimalloc initializes on this thread
def alloc():
import pyarrow as pa
pa.array(["x"] * 5000)
a = threading.Thread(target=imp); a.start(); a.join()
b = threading.Thread(target=alloc); b.start(); b.join()
```
Result: SIGSEGV (exit code 139). The same script:
- does **not** crash if `import pyarrow` happens on the main thread first;
- does **not** crash with `ARROW_DEFAULT_MEMORY_POOL=system`;
- does **not** crash on pyarrow 24.0.0.
#### Root cause
This is upstream mimalloc bug microsoft/mimalloc#1287 ("mimalloc >= 3.3.0 causes segmentation faults when used from multiple threads", closed 2026-06-20): mimalloc treats the first thread that initializes it as the process main thread; when that thread exits, subsequent threads crash in `mi_thread_init`/`_mi_malloc_generic`. mimalloc 3.2.x is unaffected.
Arrow 25.0.0 is the first release affected because GH-49772 / PR #49801 bumped `ARROW_MIMALLOC_BUILD_VERSION` from v3.2.7 to v3.3.1 (confirmed in `cpp/thirdparty/versions.txt` at tag `apache-arrow-25.0.0`).
The upstream fix exists only on mimalloc's `dev3` branch (commit microsoft/mimalloc@b92c116b67d0, "allow initial main thread to terminate before the process terminates (see issue #1287)"); the latest tagged mimalloc release (v3.3.2, 2026-04-29) predates it, so there is currently no tagged mimalloc version Arrow could bump to.
#### Real-world impact
This import pattern is not exotic — it is structurally guaranteed by Streamlit + pandas 3:
- pandas 3.x imports pyarrow eagerly (default pyarrow-backed `str` dtype);
- Streamlit executes user scripts (including their imports) on short-lived `ScriptRunner` threads, never on the main thread.
So in any Streamlit app using pandas 3 + pyarrow 25, libarrow first loads on script-run thread #1, and the next script run (a simple page reload) segfaults the whole server on its first string-column conversion (`pandas maybe_convert_objects` → `arrow::py::NdarrayToArrow`). We observed 7+ identical crashes per day on one such app before diagnosing.
#### Workarounds
- `ARROW_DEFAULT_MEMORY_POOL=system` (verified);
- `import pyarrow` on the main thread before any worker thread touches it (verified);
- pin `pyarrow<=24.0.0` (bundles unaffected mimalloc v3.2.7).
#### Suggested resolution
Consider reverting the bundled mimalloc to the v3.2.x line until a tagged mimalloc release contains the #1287 fix, or picking up that fix as a patch on top of v3.3.1.
Related but distinct: #50428 (interpreter-teardown crash when co-loading another mimalloc-v3-bundling extension; the `ARROW_DEFAULT_MEMORY_POOL=system` workaround reportedly does not help there, whereas it does here).
#### Environment
- pyarrow 25.0.0 (PyPI wheel), pandas 3.0.3, streamlit 1.59.1
- CPython 3.14.5 (Homebrew), macOS 26 (Darwin 25.5.0), Apple Silicon
- Note: mimalloc#1287 was reproduced upstream on Linux and Windows as well, so this is unlikely to be macOS-specific.
### Component(s)
C++, Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Python] SIGSEGV in bundled mimalloc mi_thread_init when libarrow is first loaded on a non-main thread that exits (mimalloc 3.3.x, pyarrow 25.0.0) [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #50471:
URL: https://github.com/apache/arrow/issues/50471#issuecomment-5022721184
I've been able to reproduce this issue consistently with the minimal reproducer shared above. Thanks @L3o-pold for it.
Bumping mimalloc with the newly released version (3.4.1) and building a new wheel solves the issue. I haven't been able to reproduce it with the new wheel anymore. I've pushed a PR bumping mimalloc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Python] SIGSEGV in bundled mimalloc mi_thread_init when libarrow is first loaded on a non-main thread that exits (mimalloc 3.3.x, pyarrow 25.0.0) [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #50471:
URL: https://github.com/apache/arrow/issues/50471#issuecomment-4955429317
I haven't investigated yet so I don't fully understand the issue or the solution but from my initial understanding we could add to our `ci/vcpkg/ports.patch` the patch from the upstream commit https://github.com/microsoft/mimalloc/commit/b92c116b67d0 and build the wheels with a patched mimalloc. There were some discussions about a possible 25.0.1 patch release but it's not entirely clear when those patches will be ready.
@kou @pitrou any thoughts on the issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Python] SIGSEGV in bundled mimalloc mi_thread_init when libarrow is first loaded on a non-main thread that exits (mimalloc 3.3.x, pyarrow 25.0.0) [arrow]
Posted by "lukasmasuch (via GitHub)" <gi...@apache.org>.
lukasmasuch commented on issue #50471:
URL: https://github.com/apache/arrow/issues/50471#issuecomment-4951313512
Streamlit dev here 👋 The PyArrow 25.0.0 update broke our CI and very likely also many apps that get updated to this version. Is there a way to get this patched quickly? cc @raulcd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Python] SIGSEGV in bundled mimalloc mi_thread_init when libarrow is first loaded on a non-main thread that exits (mimalloc 3.3.x, pyarrow 25.0.0) [arrow]
Posted by "gkotian (via GitHub)" <gi...@apache.org>.
gkotian commented on issue #50471:
URL: https://github.com/apache/arrow/issues/50471#issuecomment-5003137160
We hit this same crash in production and can confirm it from a different
platform and a different trigger path than the original report.
**Environment:** Linux x86_64 (`python:3.14-slim` Debian container on
Kubernetes; also reproduced on a local Linux machine), Python 3.14,
pyarrow 25.0.0, polars 1.42.1, pandas 2.3.3, Streamlit 1.53.1.
**Trigger:** not the pandas 3 eager-import path described above — we are on
pandas 2.3.3. Our entry point is `polars.DataFrame.to_pandas()` inside a
Streamlit app: libarrow is first imported on a short-lived Streamlit
`ScriptRunner` thread, and the crash fires when the conversion
(`pyarrow.Table.to_pandas` → `table_to_dataframe`) performs its first
allocation on an Arrow C++ thread-pool worker. Same mechanism, different
entry point — so it is not specific to pandas 3 or `NdarrayToArrow`.
Faulthandler output (trimmed):
```
Fatal Python error: Segmentation fault
Current thread 0x00007fe72bfff6c0 [ScriptRunner.sc] (most recent call first):
<no Python frame> # Arrow C++ pool worker; thread name inherited from creator
Thread 0x00007fe6457ff6c0 [ScriptRunner.sc] (most recent call first):
File "site-packages/pyarrow/pandas_compat.py", line 827 in table_to_dataframe
File "site-packages/polars/dataframe/frame.py", line 2691 in _to_pandas_without_object_columns
File "site-packages/polars/dataframe/frame.py", line 2629 in to_pandas
...
Current thread's C stack trace (most recent call first):
libarrow.so.2500 +0x16f8b89 / +0x16feaa1 / +0x16ee5c7
libarrow.so.2500 +0xb953bb / +0xb9bd9c / +0xb9be29
libarrow.so.2500 arrow::AllocateBuffer(long long, long long, arrow::MemoryPool*)+0x35
libarrow.so.2500 arrow::AllocateBuffer(long long, arrow::MemoryPool*)+0x11
libarrow_python.so.2500 +0x5fc70 / +0x79cfb / +0x5d705 / +0x5f96d / +0x5eb5e / +0x6f2c4
libarrow.so.2500 +0xbdb7f6 / +0xbdb1c4 # thread pool task loop
libstdc++.so.6 +0xeb919 # std::thread
```
Operational impact matches the report: the whole Streamlit server dies with
SIGSEGV and no Python traceback. On Kubernetes this presents as unexplained
pod crash-loops immediately after deployment (first page loads), which took
some effort to attribute.
**Workaround confirmed:** `ARROW_DEFAULT_MEMORY_POOL=system` — deployed to
production, no crashes since (previously two crashes within the first two
minutes of the pod's life).
**Possibly useful for resolving this:** mimalloc **v3.4.1** (tagged
2026-07-15) now contains the fix commit
microsoft/mimalloc@b92c116b67d0 — I verified the commit is an ancestor of
the `v3.4.1` tag. So a tagged mimalloc release containing the fix exists
now, which was not the case when this issue was opened; bumping
`ARROW_MIMALLOC_BUILD_VERSION` may be viable for a 25.0.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Python] SIGSEGV in bundled mimalloc mi_thread_init when libarrow is first loaded on a non-main thread that exits (mimalloc 3.3.x, pyarrow 25.0.0) [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #50471:
URL: https://github.com/apache/arrow/issues/50471#issuecomment-4964105930
I don't object that we backport the patch by ourselves but I'm negative for the approach because it may introduce another problem.
How about using system `malloc()` on non-main thread something like the following?
```diff
diff --git a/cpp/src/arrow/memory_pool.cc b/cpp/src/arrow/memory_pool.cc
index 0b4843bec3..d00760291d 100644
--- a/cpp/src/arrow/memory_pool.cc
+++ b/cpp/src/arrow/memory_pool.cc
@@ -130,12 +130,40 @@ std::optional<MemoryPoolBackend> UserSelectedBackend() {
return user_selected_backend;
}
+bool IsMainThread() {
+#ifdef __linux__
+ // The main thread's ID must equal to process ID on Linux.
+ return gettid() == getpid();
+#elif defined(__APPLE__) || defined(BSD)
+ // We can use pthread_main_np() on BSD family.
+ return pthread_main_np() == 1;
+#elif defined(_WIN32)
+ // Windows doesn't provide an API for this. We can implement a
+ // heuristic approach.
+ return true;
+#else
+ // Assume the current thread is the main thread on others.
+ return true;
+#endif
+}
+
MemoryPoolBackend DefaultBackend() {
auto backend = UserSelectedBackend();
if (backend.has_value()) {
return backend.value();
}
- struct SupportedBackend default_backend = SupportedBackends().front();
+ static struct SupportedBackend default_backend;
+ static std::once_flag once;
+ std::call_once(once, [&]() {
+ if (IsMainThread()) {
+ default_backend = SupportedBackends().front();
+ } else {
+ // Fallback to the system backend.
+ // We can remove this once mimalloc()'s thread related problem is fixed.
+ // See also: https://github.com/microsoft/mimalloc/issues/1287
+ default_backend = SupportedBackends().back();
+ }
+ });
return default_backend.backend;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Python] SIGSEGV in bundled mimalloc mi_thread_init when libarrow is first loaded on a non-main thread that exits (mimalloc 3.3.x, pyarrow 25.0.0) [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #50471:
URL: https://github.com/apache/arrow/issues/50471#issuecomment-5020675468
> I don't object that we backport the patch by ourselves but I'm negative for the approach because it may introduce another problem.
We can simply bump mimalloc to the just-released 3.4.1. Also @raulcd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Python] SIGSEGV in bundled mimalloc mi_thread_init when libarrow is first loaded on a non-main thread that exits (mimalloc 3.3.x, pyarrow 25.0.0) [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd closed issue #50471: [C++][Python] SIGSEGV in bundled mimalloc mi_thread_init when libarrow is first loaded on a non-main thread that exits (mimalloc 3.3.x, pyarrow 25.0.0)
URL: https://github.com/apache/arrow/issues/50471
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Python] SIGSEGV in bundled mimalloc mi_thread_init when libarrow is first loaded on a non-main thread that exits (mimalloc 3.3.x, pyarrow 25.0.0) [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #50471:
URL: https://github.com/apache/arrow/issues/50471#issuecomment-5032125261
Bundled mimalloc has been merged here:
- https://github.com/apache/arrow/issues/50428
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org