You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "eligriv75 (via GitHub)" <gi...@apache.org> on 2026/07/10 09:59:00 UTC
[I] R crash when calling arrow::write_parquet() [arrow]
eligriv75 opened a new issue, #50466:
URL: https://github.com/apache/arrow/issues/50466
### Describe the bug, including details regarding any error messages, version, and platform.
### System details
RStudio Edition : Desktop
RStudio Version : 2026.06.0
OS Version : Windows Server 2019
R Version : 4.6.1
### Steps to reproduce the problem
R code :
```
pkg <- c(
"tidyverse",
"data.table",
"haven",
"stringr",
"readxl",
"openxlsx",
"janitor",
"glue",
"knitr",
"readr",
"lintr",
"sqldf",
"gtools",
"jsonlite",
"sampling",
"ggformula",
"arrow"
)
sapply(pkg, library, character.only = TRUE)
sessionInfo()
write_parquet(iris, "test.parquet")
```
The sessionInfo() output is :
```
R version 4.6.1 (2026-06-24 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows Server 2019 x64 (build 17763)
Matrix products: default
LAPACK version 3.12.1
locale:
[1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252
[4] LC_NUMERIC=C LC_TIME=French_France.1252
time zone: Europe/Paris
tzcode source: internal
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] arrow_24.0.0 ggformula_1.0.1 ggridges_0.5.7 ggiraph_0.9.6 scales_1.4.0
[6] sampling_2.11 jsonlite_2.0.0 gtools_3.9.5 sqldf_0.4-12 RSQLite_3.53.3
[11] gsubfn_0.7 proto_1.0.0 lintr_3.3.0-1 knitr_1.51 glue_1.8.1
[16] janitor_2.2.1 openxlsx_4.2.8.1 readxl_1.5.0 haven_2.5.5 data.table_1.18.4
[21] lubridate_1.9.5 forcats_1.0.1 stringr_1.6.0 dplyr_1.2.1 purrr_1.2.2
[26] readr_2.2.0 tidyr_1.3.2 tibble_3.3.1 ggplot2_4.0.3 tidyverse_2.0.0
loaded via a namespace (and not attached):
[1] gtable_0.3.6 xfun_0.59 htmlwidgets_1.6.4 tzdb_0.5.0
[5] vctrs_0.7.3 tools_4.6.1 generics_0.1.4 chron_2.3-62
[9] blob_1.3.0 pkgconfig_2.0.3 RColorBrewer_1.1-3 S7_0.2.2
[13] assertthat_0.2.1 lifecycle_1.0.5 compiler_4.6.1 farver_2.1.2
[17] snakecase_0.11.1 fontLiberation_0.1.0 fontquiver_0.2.1 htmltools_0.5.9
[21] pillar_1.11.1 MASS_7.3-65 cachem_1.1.0 mosaicCore_0.9.5
[25] fontBitstreamVera_0.1.1 digest_0.6.39 tidyselect_1.2.1 zip_3.0.0
[29] stringi_1.8.7 labelled_2.16.0 fastmap_1.2.0 grid_4.6.1
[33] cli_3.6.6 magrittr_2.0.5 withr_3.0.3 gdtools_0.5.1
[37] backports_1.5.1 bit64_4.8.2 timechange_0.4.0 bit_4.6.0
[41] otel_0.2.0 cellranger_1.1.0 hms_1.1.4 lpSolve_5.6.23
[45] memoise_2.0.1 evaluate_1.0.5 tcltk_4.6.1 rex_1.2.2
[49] rlang_1.3.0 Rcpp_1.1.2 DBI_1.3.0 xml2_1.6.0
[53] rstudioapi_0.19.0 R6_2.6.1 systemfonts_1.3.2
```
### Describe the problem in detail
When loading a relatively large number of packages, calls to certain arrow functions cause RStudio to crash with the error message “R session aborted” and a bomb icon.
I don't quite understand why the crash occurs, or why my workarounds work. For now, there are several ways I can get around the problem. The first, for example, is to add a `read_parquet()` before my `write_parquet()`.
Another is to simply remove or add randomly some packages from the list of loaded packages. Sometines it does fix the issue, sometimes it doesn't. I haven’t tried all the combinations that cause it to crash or not.
The same code but with this package list (I just add xlsx compared to the previous list) instead does not crash :
```
pkg <- c(
"tidyverse",
"data.table",
"haven",
"stringr",
"readxl",
"openxlsx",
"xlsx",
"janitor",
"glue",
"knitr",
"readr",
"lintr",
"sqldf",
"gtools",
"jsonlite",
"sampling",
"ggformula",
"arrow"
)
```
The bug also occurs when running the code in Rgui. However, it works fine in RStudio Server installed on a Linux environment. That’s why I’m reaching out to you to try to resolve the issue because the cause may be due to arrow.
Another finding : the behavior is exactly the same in R4.5.3 with arrow 24.0.0 but in R4.4.3 with arrow 23.0.1.2 there is no problem.
### Describe the behavior you expected
The program should write a basic parquet file as output.
### Component(s)
R
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] R crash when calling arrow::write_parquet() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #50466:
URL: https://github.com/apache/arrow/issues/50466#issuecomment-5059879639
@eligriv75 - Let me know if you get the chance to test this with the new Arrow release, and if it's now fixed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] R crash when calling arrow::write_parquet() [arrow]
Posted by "eligriv75 (via GitHub)" <gi...@apache.org>.
eligriv75 commented on issue #50466:
URL: https://github.com/apache/arrow/issues/50466#issuecomment-4982099555
`Sys.setenv(ARROW_DEFAULT_MEMORY_POOL = "system")` does indead clear the error. Thanks!
That's good news. Do you know roughly when this release will be available?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] R crash when calling arrow::write_parquet() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #50466:
URL: https://github.com/apache/arrow/issues/50466#issuecomment-4981998326
Would you mind trying with `Sys.setenv(ARROW_DEFAULT_MEMORY_POOL = "system")` **before** Arrow is loaded (this is important, as once it's loaded that environment variable won't have an effect) to see if it still crashes?
I think it might be due to mimalloc, one of our dependencies we bundle with Arrow, and if that's right then the new Arrow version we're about to release should solve it 🤞
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] R crash when calling arrow::write_parquet() [arrow]
Posted by "eligriv75 (via GitHub)" <gi...@apache.org>.
eligriv75 commented on issue #50466:
URL: https://github.com/apache/arrow/issues/50466#issuecomment-5060424419
Hi @thisisnic , your message comes at just the right time, since we were actually in the middle of testing the new Arrow release. Based on our reproducible examples of the bug, the release appears to resolve the issue. I didn't see this issue listed among the bug fixes, but given the results, I imagine it has indeed been included. I’ll pass this news on to our users, and if we receive any negative feedback, I’ll let you know. Thanks again for your help and for your quick response!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] R crash when calling arrow::write_parquet() [arrow]
Posted by "CelestineSauvage (via GitHub)" <gi...@apache.org>.
CelestineSauvage commented on issue #50466:
URL: https://github.com/apache/arrow/issues/50466#issuecomment-4981806906
Hi, I work with @eligriv75 😄
> "Also, out of interest, does running options(arrow.use_threads = FALSE) before write_parquet() change anything?"
— the crash persists.
I have run the debug both with and without this option; here are the logs for the single-threaded run:"
```
(1510.b04): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
arrow!Z17Array__infer_typeP7SEXPREC+0x119f72e:
00007ffa`f75090be 498b4308 mov rax,qword ptr [r11+8] ds:00000000`00000008=????????????????
0:000> k
# Child-SP RetAddr Call Site
00 00000037`517f6080 00007ffa`f7230d2b arrow!Z17Array__infer_typeP7SEXPREC+0x119f72e
01 00000037`517f60c0 00007ffa`f7d5361e arrow!Z17Array__infer_typeP7SEXPREC+0xec739b
02 00000037`517f6150 00007ffa`f7dae4fa arrow!ZN12GcMemoryPool8AllocateExxPPh+0x3e
03 00000037`517f6220 00007ffa`f7dade4d arrow!ZN5arrow10MemoryPool13ReleaseUnusedEv+0x91a
04 00000037`517f62c0 00007ffa`f7233ec5 arrow!ZN5arrow10MemoryPool13ReleaseUnusedEv+0x26d
05 00000037`517f6370 00007ffa`f7db58b5 arrow!Z17Array__infer_typeP7SEXPREC+0xeca535
06 00000037`517f63d0 00007ffa`f807469e arrow!ZN5arrow13BufferBuilder6ResizeExb+0x95
07 00000037`517f6430 00007ffa`f8087212 arrow!ZN5arrow8internal14SerialExecutor19RunInSerialExecutorIbNS_6FutureIbEENS_6ResultIbEEEET1_NS0_6FnOnceIFNS3_IT_EEPNS0_8ExecutorEEEE+0x155e
08 00000037`517f6500 00007ffa`f6efc2b4 arrow!ZN5arrow8internal19DictionaryConverterINS_9UInt8TypeENS_1r10RConverterEE4InitEPNS_10MemoryPoolE+0x10f2
09 00000037`517f65f0 00007ffa`f7e61941 arrow!Z17Array__infer_typeP7SEXPREC+0xb92924
0a 00000037`517f6640 00007ffa`f7dff2e8 arrow!ZN5arrow1r9Converter7ConvertERKSt10shared_ptrINS_5ArrayEE+0x36c1
0b 00000037`517f66a0 00007ffa`f6dd36ac arrow!ZN5arrow15BaseListBuilderINS_8ListTypeEED1Ev+0x2ba88
0c 00000037`517f6710 00007ffa`f8085240 arrow!Z17Array__infer_typeP7SEXPREC+0xa69d1c
0d 00000037`517f6770 00007ffa`f81c206c arrow!ZN5arrow8internal19DictionaryConverterINS_10StringTypeENS_1r10RConverterEE4InitEPNS_10MemoryPoolE+0x40
0e 00000037`517f67f0 00007ffa`f8081854 arrow!ZN5arrow8internal9ConverterIP7SEXPRECNS_1r18RConversionOptionsEE9ConstructESt10shared_ptrINS_8DataTypeEES5_PNS_10MemoryPoolE+0x6c
0f 00000037`517f6830 00007ffa`f7e00fde arrow!ZN5arrow8internal17MakeConverterImplINS_1r10RConverterENS2_15RConverterTraitEE5VisitERKNS_14DictionaryTypeE+0xc4
10 00000037`517f68e0 00007ffa`f6344edb arrow!ZN5arrow15VisitTypeInlineINS_8internal17MakeConverterImplINS_1r10RConverterENS3_15RConverterTraitEEEJEEENS_6StatusERKNS_8DataTypeEPT_DpOT0_+0x10ce
11 00000037`517f6960 00007ffa`f634631f arrow!ZN5arrow1r26vec_to_arrow__reuse_memoryEP7SEXPREC+0xfb
12 00000037`517f69f0 00007ffa`f62e1df7 arrow!Z16Table__from_dotsP7SEXPRECS0_b+0xb0f
13 00000037`517f6c80 00007ffb`1c72227e arrow!arrow_Table__from_dots+0x57
14 00000037`517f8d00 00007ffb`1c5dcf35 R!rwarn_+0x28e0e
15 00000037`517f8f30 00007ffb`1c5ebe6b R!R_ParseEvalString+0x3fc5
16 00000037`517f9840 00007ffb`1c5ec15b R!R_ParseEvalString+0x12efb
17 00000037`517f9910 00007ffb`1c5f1282 R!R_ParseEvalString+0x131eb
18 00000037`517f9bf0 00007ffb`1c5f263f R!Rf_eval+0x4a62
19 00000037`517f9ff0 00007ffb`1c5f32bb R!Rf_eval+0x5e1f
1a 00000037`517fa080 00007ffb`1c630b12 R!Rf_eval+0x6a9b
1b 00000037`517fa0c0 00007ffb`1c6317eb R!getPRIMNAME+0x932
1c 00000037`517fa140 00007ffb`1c631f5c R!R_has_methods_attached+0x37b
1d 00000037`517fc1c0 00007ffb`1c63215d R!R_has_methods_attached+0xaec
1e 00000037`517fc270 00007ffb`1c5e1072 R!R_has_methods_attached+0xced
1f 00000037`517fc720 00007ffb`1c5ebe6b R!R_ParseEvalString+0x8102
20 00000037`517fd030 00007ffb`1c5ec15b R!R_ParseEvalString+0x12efb
21 00000037`517fd100 00007ffb`1c5f1282 R!R_ParseEvalString+0x131eb
22 00000037`517fd3e0 00007ffb`1c5f263f R!Rf_eval+0x4a62
23 00000037`517fd7e0 00007ffb`1c5ec36d R!Rf_eval+0x5e1f
24 00000037`517fd870 00007ffb`1c6188fd R!R_ParseEvalString+0x133fd
25 00000037`517fdb50 00007ffb`1c618c70 R!Rf_ReplIteration+0x26d
26 00000037`517fdbb0 00007ffb`1c618d12 R!Rf_ReplIteration+0x5e0
27 00000037`517fec30 00007ff6`81b4b5a3 R!run_Rmainloop+0x52
28 00000037`517fec60 00007ff6`81b44db4 rsession_utf8!rd_traceback+0x9fbb43
29 00000037`517fef00 00007ff6`80ec7101 rsession_utf8!rd_traceback+0x9f5354
2a 00000037`517ff160 00007ff6`820b9270 rsession_utf8!git_trace_set+0x1e9501
2b 00000037`517ffd80 00007ffb`4f3b7ac4 rsession_utf8!rd_traceback+0xf69810
2c 00000037`517ffdc0 00007ffb`4fdba8c1 KERNEL32!BaseThreadInitThunk+0x14
2d 00000037`517ffdf0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] R crash when calling arrow::write_parquet() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #50466:
URL: https://github.com/apache/arrow/issues/50466#issuecomment-5060628838
FWIW I think it was fixed by #49801 which upgraded the version of the mimalloc library that we bundle with Arrow - the issue came from that library but were fixed in a newer version.
I'll close this as resolved, but if you have any new problems, please open a new issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] R crash when calling arrow::write_parquet() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic closed issue #50466: [R] R crash when calling arrow::write_parquet()
URL: https://github.com/apache/arrow/issues/50466
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] R crash when calling arrow::write_parquet() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #50466:
URL: https://github.com/apache/arrow/issues/50466#issuecomment-4963193034
Thanks for reporting this @eligriv75! One thing that could help us get to the bottom of this is a stack trace on that crash. If you're able to install [WinDbg](https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/), we could use that. I generated some instructions, show below.
1. Install [WinDbg](https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/) from the Microsoft Store (or via winget install Microsoft.WinDbg).
2. Start RStudio and load your packages as usual, stopping just before the write_parquet() call.
3. In WinDbg: File → Attach to process → select rsession.exe (there may be more than one — pick the one whose start time matches your session, or close other RStudio windows first).
4. Once attached, the process will be paused — press g (or click Go/Continue) to let it run again.
5. Back in RStudio, run the write_parquet() call to trigger the crash. WinDbg should break in automatically.
6. In the WinDbg command window, run k (or ~*k for all threads) and paste the output here, along with the exception line WinDbg prints when it breaks (e.g. an access violation code).
Also, out of interest, does running `options(arrow.use_threads = FALSE)` before `write_parquet()` change anything?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] R crash when calling arrow::write_parquet() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #50466:
URL: https://github.com/apache/arrow/issues/50466#issuecomment-4982149653
OK, that option is a good workaround, though you might see reduced performance depending on dataset size.
Release will be in the next couple of weeks - next few days if everything goes well and a little longer if not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] R crash when calling arrow::write_parquet() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #50466:
URL: https://github.com/apache/arrow/issues/50466#issuecomment-4982488996
Thanks again both for the detailed responses - these things are a pain for us to get to the bottom of sometimes, so you being up for trying different things here :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org