You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "PMassicotte (via GitHub)" <gi...@apache.org> on 2023/02/15 22:23:17 UTC

[GitHub] [arrow] PMassicotte opened a new issue, #34211: [R] Segmentation fault when using write_parquet()

PMassicotte opened a new issue, #34211:
URL: https://github.com/apache/arrow/issues/34211

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   I am randomly getting segfault when using `write_parquet()` with the latest release (the same code works well with v 10.0.1). 
   
   ```
    *** caught segfault ***
   address 0x18, cause 'memory not mapped'
   
   Traceback:
    1: Table__from_dots(dots, schema, option_use_threads())
    2: Table$create(x, schema = schema)
    3: as_arrow_table.data.frame(x)
    4: as_arrow_table(x)
    5: doTryCatch(return(expr), name, parentenv, handler)
    6: tryCatchOne(expr, names, parentenv, handlers[[1L]])
    7: tryCatchList(expr, classes, parentenv, handlers)
    8: tryCatch(as_arrow_table(x), arrow_no_method_as_arrow_table = function(e) {    abort("Object must be coercible to an Arrow Table using `as_arrow_table()`",         parent = e, call = caller_env(2))})
    9: as_writable_table(x)
   10: write_parquet(bioargo_dark_corrected, here("data", "raw", "bioargo",     "bioargo_correction_c.parquet"))
   11: eval(ei, envir)
   12: eval(ei, envir)
   13: withVisible(eval(ei, envir))
   14: source(here("R", "001c_bioargo_chla_dark_correction.R"))
   
   Possible actions:
   1: abort (with core dump, if enabled)
   2: normal R exit
   3: exit R without saving workspace
   4: exit R saving workspace
   ```
   
   Following this (https://arrow.apache.org/docs/7.0/r/articles/developers/debugging.html), here is the exact line when the code crashes.
   
   
   ```
   Thread 1 "R" received signal SIGSEGV, Segmentation fault.
   0x00007fffdccb0956 in std::__shared_ptr<arrow::DataType, (__gnu_cxx::_Lock_policy)2>::__shared_ptr (this=0x7ffffffc4d50) at /usr/include/c++/12/bits/shared_ptr_base.h:1522
   1522          __shared_ptr(const __shared_ptr&) noexcept = default;
   ````
   
   
   ```r
   $> sessionInfo()
   R version 4.2.2 (2022-10-31)
   Platform: x86_64-pc-linux-gnu (64-bit)
   Running under: Ubuntu 22.10
   
   Matrix products: default
   BLAS/LAPACK: /opt/intel/compilers_and_libraries_2020.4.304/linux/mkl/lib/intel64_lin/libmkl_gf_lp64.so
   
   locale:
    [1] LC_CTYPE=en_CA.UTF-8       LC_NUMERIC=C               LC_TIME=en_CA.UTF-8       
    [4] LC_COLLATE=en_CA.UTF-8     LC_MONETARY=en_CA.UTF-8    LC_MESSAGES=en_CA.UTF-8   
    [7] LC_PAPER=en_CA.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
   [10] LC_TELEPHONE=C             LC_MEASUREMENT=en_CA.UTF-8 LC_IDENTIFICATION=C       
   
   attached base packages:
   [1] stats     graphics  grDevices utils     datasets  methods   base     
   
   other attached packages:
    [1] gsw_1.1-1              fishmethods_1.12-0     santoku_0.9.0          arrow_11.0.0.100000089
    [5] yardstick_1.1.0        workflowsets_1.0.0     workflows_1.1.2        tune_1.0.1            
    [9] rsample_1.1.1          recipes_1.0.4          parsnip_1.0.3          modeldata_1.1.0       
   [13] infer_1.0.4            dials_1.1.0            scales_1.2.1           broom_1.0.3           
   [17] tidymodels_1.0.0       data.table_1.14.6      furrr_0.3.1            future_1.31.0         
   [21] pins_1.1.0             tidyterra_0.3.1        terra_1.7-3            sf_1.0-9              
   [25] patchwork_1.1.2        tidync_0.3.0           here_1.0.1             glue_1.6.2            
   [29] ggpmthemes_0.0.2       lubridate_1.9.2        forcats_1.0.0          stringr_1.5.0         
   [33] dplyr_1.1.0            purrr_1.0.1            readr_2.1.4            tidyr_1.3.0           
   [37] tibble_3.1.8           ggplot2_3.4.1          tidyverse_1.3.2.9000  
   
   loaded via a namespace (and not attached):
    [1] minqa_1.2.5         colorspace_2.1-0    ellipsis_0.3.2      class_7.3-21       
    [5] rprojroot_2.0.3     fs_1.6.1            rstudioapi_0.14     proxy_0.4-27       
    [9] farver_2.1.1        listenv_0.9.0       bit64_4.0.5         prodlim_2019.11.13 
   [13] fansi_1.0.4         codetools_0.2-19    splines_4.2.2       ncdf4_1.21         
   [17] extrafont_0.19      jsonlite_1.8.4      nloptr_2.0.3        Rttf2pt1_1.3.12    
   [21] compiler_4.2.2      backports_1.4.1     assertthat_0.2.1    Matrix_1.5-3       
   [25] cli_3.6.0           tools_4.2.2         gtable_0.3.1        rappdirs_0.3.3     
   [29] Rcpp_1.0.10         RNetCDF_2.6-2       DiceDesign_1.9      vctrs_0.5.2        
   [33] nlme_3.1-162        extrafontdb_1.0     iterators_1.0.14    timeDate_4022.108  
   [37] gower_1.0.1         globals_0.16.2      lme4_1.1-31         timechange_0.2.0   
   [41] lifecycle_1.0.3     ncmeta_0.3.5        MASS_7.3-58.2       ipred_0.9-13       
   [45] hms_1.1.2           parallel_4.2.2      TMB_1.9.2           rpart_4.1.19       
   [49] stringi_1.7.12      foreach_1.5.2       e1071_1.7-13        lhs_1.1.6          
   [53] boot_1.3-28.1       hardhat_1.2.0       lava_1.7.1          rlang_1.0.6        
   [57] pkgconfig_2.0.3     lattice_0.20-45     labeling_0.4.2      bit_4.0.5          
   [61] tidyselect_1.2.0    parallelly_1.34.0   magrittr_2.0.3      R6_2.5.1           
   [65] generics_0.1.3      bootstrap_2019.6    DBI_1.1.3           pillar_1.8.1       
   [69] withr_2.5.0         units_0.8-1         survival_3.5-3      nnet_7.3-18        
   [73] future.apply_1.10.0 crayon_1.5.2        KernSmooth_2.23-20  utf8_1.2.3         
   [77] tzdb_0.3.0          grid_4.2.2          digest_0.6.31       classInt_0.4-8     
   [81] numDeriv_2016.8-1.1 GPfit_1.0-8         munsell_0.5.0     
   ```
   
   ### Component(s)
   
   R


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] thisisnic commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1432792056

   Thanks for reporting this @PMassicotte.  A couple of things to suggest:
   1. Does uninstalling and reinstalling the package help at all?
   2. Would you mind running `arrow::arrow_info()` and sharing the output?
   3. Is it possible to narrow things down to a small reproducible example? It'll make it easier to work out exactly what's going on here.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] PMassicotte commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "PMassicotte (via GitHub)" <gi...@apache.org>.
PMassicotte commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1432997803

   Thank you @thisisnic for replying. 
   
   1. I just tried, but no luck.
   
   2.
   
   ```r
   r$> arrow::arrow_info()
   Arrow package version: 11.0.0.2
   
   Capabilities:
                  
   dataset    TRUE
   substrait FALSE
   parquet    TRUE
   json       TRUE
   s3         TRUE
   gcs        TRUE
   utf8proc   TRUE
   re2        TRUE
   snappy     TRUE
   gzip       TRUE
   brotli     TRUE
   zstd       TRUE
   lz4        TRUE
   lz4_frame  TRUE
   lzo       FALSE
   bz2        TRUE
   jemalloc   TRUE
   mimalloc   TRUE
   
   Memory:
                      
   Allocator  jemalloc
   Current     0 bytes
   Max       112.63 Mb
   
   Runtime:
                           
   SIMD Level          avx2
   Detected SIMD Level avx2
   
   Build:
                              
   C++ Library Version  11.0.0
   C++ Compiler            GNU
   C++ Compiler Version  7.5.0
   ```
   
   3. I will try to create a subset of my data that create the same error.
   
   Thank you very much.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] thisisnic commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1457987626

   @egillax Thanks for reporting this.
   
   @PMassicotte Would you mind trying again with the debugger attached, but pasting the complete output of when it crashes?  Sometimes the top line isn't quite the right one to give us enough clues, and I want to see if your error is the same as the one reported by @egillax .
   
   @paleolimbot Would you mind taking a look at this? If this is a common error between both of the folks above, the line that seems to be where the crash happens was modified in #14277, so you'll know more than me about that bit.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] PMassicotte commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "PMassicotte (via GitHub)" <gi...@apache.org>.
PMassicotte commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458508211

   I think I was able to reproduce it.
   
   ```r
   library(tidyverse)
   library(arrow)
   
   
   file <- curl::curl_download("https://download849.mediafire.com/r4csstfcwwwgGquvCho4H6GtScoCJac108RL-q6X9MtoWuPDQvZOQAWhxQqlCjLj2RmsyzikhTZ0ijBElIAs5in5whbp-w/7dk60h8gnj4n1qj/bioargo_correction_b.parquet", destfile = tempfile(fileext = ".parquet"))
   
   bioargo <- read_parquet(
     file
   )
   
   bioargo
   
   bioargo |>
     group_by(takuse, date_time, n_prof) |>
     filter(pres == max(pres)) |>
     ggplot(aes(x = pres)) +
     geom_histogram(binwidth = 10, color = "white")
   
   bioargo_dark_corrected <- bioargo |>
     group_by(takuse, date_time, n_prof) |>
     mutate(chla = chla - min(chla, na.rm = TRUE)) |>
     ungroup()
   
   write_parquet(
     bioargo_dark_corrected,
     tempdir()
   )
   
   ```
   
   Can you confirm?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458562241

   It looks like this is a problem with ALTREP bypass: the array (probably most of them) are already ALTREP coming from Arrow, and the segfault we get is when we try to get the chunked array back:
   
   ```
   Console is in 'commands' mode, prefix expressions with '?'.
   Attached to process 45901
   Stop reason: EXC_BAD_ACCESS (code=1, address=0x18)
   bt
   * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x18)
     * frame #0: 0x00000001128bd8a8 arrow.so`std::__1::shared_ptr<arrow::DataType>::shared_ptr(this=0x000000016fc26390, __r=nullptr) at shared_ptr.h:846:18
       frame #1: 0x00000001128b9224 arrow.so`std::__1::shared_ptr<arrow::DataType>::shared_ptr(this=0x000000016fc26390, __r=nullptr) at shared_ptr.h:848:1
       frame #2: 0x00000001128b9dd8 arrow.so`arrow::r::InferArrowType(x=0x0000000137c42438) at type_infer.cpp:189:12
       frame #3: 0x00000001128b4980 arrow.so`arrow::r::InferSchemaFromDots(this=(size=111), j=0, x=0x0000000137c42438, name="filename")::$_1::operator()(int, SEXPREC*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>) const at table.cpp:179:38
       frame #4: 0x00000001128a8740 arrow.so`void arrow::r::TraverseDots<arrow::r::InferSchemaFromDots(SEXPREC*, SEXPREC*, int, std::__1::shared_ptr<arrow::Schema>&)::$_1>(dots=cpp11::list @ 0x000000016fc266a0, num_fields=111, lambda=(size=111))::$_1) at arrow_types.h:211:9
       frame #5: 0x00000001128a8450 arrow.so`arrow::r::InferSchemaFromDots(lst=0x0000000150258fa0, schema_sxp=0x000000013500d0e0, num_fields=111, schema=nullptr) at table.cpp:182:3
       frame #6: 0x000000011277aeb0 arrow.so`Table__from_dots(lst=0x0000000150258fa0, schema_sxp=0x000000013500d0e0, use_threads=true) at r_to_arrow.cpp:1461:15
       frame #7: 0x00000001124eb658 arrow.so`::_arrow_Table__from_dots(lst_sexp=0x0000000150258fa0, 
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458372525

   Biogeochemical Argo!!!! (Cool to see it here...that's what I did in my previous job!)
   
   Since all these seem related to `InferArrowType()`, do you mind attempting to print out the column names and types of the data frame you're passing to `arrow::write_parquet()`? You could do that maybe with `str(the_data_frame_right_before_write_parquet[integer(0), ])`?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] PMassicotte commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "PMassicotte (via GitHub)" <gi...@apache.org>.
PMassicotte commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458494011

   > If I'm not mistaken, I believe you're even using a package I wrote to read the NetCDFs!
   
   Yes!
   
   I just tried:
   
   1. Write as RDS
   2. Read it back in R
   3. Write it using Arrow
   
   This is not crashing. So it looks like that the problem (data type?) disappear if I save/read using another format. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458544045

   Brilliant! I had to download the file separately but this is fantastic.
   
   ```r
   library(tidyverse)
   library(arrow)
   
   # Download from:
   # https://download849.mediafire.com/r4csstfcwwwgGquvCho4H6GtScoCJac108RL-q6X9MtoWuPDQvZOQAWhxQqlCjLj2RmsyzikhTZ0ijBElIAs5in5whbp-w/7dk60h8gnj4n1qj/bioargo_correction_b.parquet
   file <- "~/Desktop/bioargo_correction_b.parquet"
   
   bioargo <- read_parquet(file)
   
   bioargo
   
   pdf(tempfile())
   bioargo |>
     group_by(takuse, date_time, n_prof) |>
     filter(pres == max(pres)) |>
     ggplot(aes(x = pres)) +
     geom_histogram(binwidth = 10, color = "white")
   dev.off()
   
   bioargo_dark_corrected <- bioargo |>
     group_by(takuse, date_time, n_prof) |>
     mutate(chla = chla - min(chla, na.rm = TRUE)) |>
     ungroup()
   
   write_parquet(bioargo_dark_corrected, tempfile())
   ```
   
   This reprex appears to crash R.
   See standard output and standard error for more details.
   
   #### Standard output and error
   
   ``` sh
   
    *** caught segfault ***
   address 0x18, cause 'invalid permissions'
   
   Traceback:
    1: Table__from_dots(dots, schema, option_use_threads())
    2: Table$create(x, schema = schema)
    3: as_arrow_table.data.frame(x)
    4: as_arrow_table(x)
    5: doTryCatch(return(expr), name, parentenv, handler)
    6: tryCatchOne(expr, names, parentenv, handlers[[1L]])
    7: tryCatchList(expr, classes, parentenv, handlers)
    8: tryCatch(as_arrow_table(x), arrow_no_method_as_arrow_table = function(e) {    abort("Object must be coercible to an Arrow Table using `as_arrow_table()`",         parent = e, call = caller_env(2))})
    9: as_writable_table(x)
   10: write_parquet(bioargo_dark_corrected, tempfile())
   11: eval(expr, envir, enclos)
   12: eval(expr, envir, enclos)
   13: eval_with_user_handlers(expr, envir, enclos, user_handlers)
   14: withVisible(eval_with_user_handlers(expr, envir, enclos, user_handlers))
   15: withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler)
   16: doTryCatch(return(expr), name, parentenv, handler)
   17: tryCatchOne(expr, names, parentenv, handlers[[1L]])
   18: tryCatchList(expr, classes, parentenv, handlers)
   19: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call, nlines = 1L)        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        sm <- strsplit(conditionMessage(e), "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && isTRUE(getOption("show.error.messages"))) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
   20: try(f, silent = TRUE)
   21: handle(ev <- withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler))
   22: timing_fn(handle(ev <- withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler)))
   23: evaluate_call(expr, parsed$src[[i]], envir = envir, enclos = enclos,     debug = debug, last = i == length(out), use_try = stop_on_error !=         2L, keep_warning = keep_warning, keep_message = keep_message,     output_handler = output_handler, include_timing = include_timing)
   24: evaluate::evaluate(...)
   25: evaluate(code, envir = env, new_device = FALSE, keep_warning = if (is.numeric(options$warning)) TRUE else options$warning,     keep_message = if (is.numeric(options$message)) TRUE else options$message,     stop_on_error = if (is.numeric(options$error)) options$error else {        if (options$error && options$include)             0L        else 2L    }, output_handler = knit_handlers(options$render, options))
   26: in_dir(input_dir(), expr)
   27: in_input_dir(evaluate(code, envir = env, new_device = FALSE,     keep_warning = if (is.numeric(options$warning)) TRUE else options$warning,     keep_message = if (is.numeric(options$message)) TRUE else options$message,     stop_on_error = if (is.numeric(options$error)) options$error else {        if (options$error && options$include)             0L        else 2L    }, output_handler = knit_handlers(options$render, options)))
   28: eng_r(options)
   29: block_exec(params)
   30: call_block(x)
   31: process_group.block(group)
   32: process_group(group)
   33: withCallingHandlers(if (tangle) process_tangle(group) else process_group(group),     error = function(e) {        setwd(wd)        cat(res, sep = "\n", file = output %n% "")        message("Quitting from lines ", paste(current_lines(i),             collapse = "-"), " (", knit_concord$get("infile"),             ") ")    })
   34: process_file(text, output)
   35: knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
   36: rmarkdown::render(input, quiet = TRUE, envir = globalenv(), encoding = "UTF-8")
   37: (function (input) {    rmarkdown::render(input, quiet = TRUE, envir = globalenv(),         encoding = "UTF-8")})(input = base::quote("loyal-rat_reprex.R"))
   38: (function (what, args, quote = FALSE, envir = parent.frame()) {    if (!is.list(args))         stop("second argument must be a list")    if (quote)         args <- lapply(args, enquote)    .Internal(do.call(what, args, envir))})(base::quote(function (input) {    rmarkdown::render(input, quiet = TRUE, envir = globalenv(),         encoding = "UTF-8")}), base::quote(list(input = "loyal-rat_reprex.R")), envir = base::quote(<environment>),     quote = base::quote(TRUE))
   39: do.call(do.call, c(readRDS("/var/folders/p5/sxv05ml96sd1n2p3ssfhzzth0000gn/T//Rtmpod6Iee/callr-fun-768e1cc53b50"),     list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,     quote = TRUE)
   40: saveRDS(do.call(do.call, c(readRDS("/var/folders/p5/sxv05ml96sd1n2p3ssfhzzth0000gn/T//Rtmpod6Iee/callr-fun-768e1cc53b50"),     list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,     quote = TRUE), file = "/var/folders/p5/sxv05ml96sd1n2p3ssfhzzth0000gn/T//Rtmpod6Iee/callr-res-768e58b90ff1",     compress = FALSE)
   41: withCallingHandlers({    NULL    saveRDS(do.call(do.call, c(readRDS("/var/folders/p5/sxv05ml96sd1n2p3ssfhzzth0000gn/T//Rtmpod6Iee/callr-fun-768e1cc53b50"),         list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,         quote = TRUE), file = "/var/folders/p5/sxv05ml96sd1n2p3ssfhzzth0000gn/T//Rtmpod6Iee/callr-res-768e58b90ff1",         compress = FALSE)    flush(stdout())    flush(stderr())    NULL    invisible()}, error = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        
 e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/var/folders/p5/sxv05ml96sd1n2p3ssfhzzth0000gn/T//Rtmpod6Iee/callr-res-768e58b90ff1",             ".error"))    }}, interrupt = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) { 
            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/var/folders/p5/sxv05ml96sd1n2p3ssfhzzth0000gn/T//Rtmpod6Iee/callr-res-768e58b90ff1",             ".error"))    }}, callr_message = function(e) {    try(signalCondition(e))})
   42: doTryCatch(return(expr), name, parentenv, handler)
   43: tryCatchOne(expr, names, parentenv, handlers[[1L]])
   44: tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
   45: doTryCatch(return(expr), name, parentenv, handler)
   46: tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]),     names[nh], parentenv, handlers[[nh]])
   47: tryCatchList(expr, classes, parentenv, handlers)
   48: tryCatch(withCallingHandlers({    NULL    saveRDS(do.call(do.call, c(readRDS("/var/folders/p5/sxv05ml96sd1n2p3ssfhzzth0000gn/T//Rtmpod6Iee/callr-fun-768e1cc53b50"),         list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,         quote = TRUE), file = "/var/folders/p5/sxv05ml96sd1n2p3ssfhzzth0000gn/T//Rtmpod6Iee/callr-res-768e58b90ff1",         compress = FALSE)    flush(stdout())    flush(stderr())    NULL    invisible()}, error = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2)
 )        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/var/folders/p5/sxv05ml96sd1n2p3ssfhzzth0000gn/T//Rtmpod6Iee/callr-res-768e58b90ff1",             ".error"))    }}, interrupt = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na
 (cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/var/folders/p5/sxv05ml96sd1n2p3ssfhzzth0000gn/T//Rtmpod6Iee/callr-res-768e58b90ff1",             ".error"))    }}, callr_message = function(e) {    try(signalCondition(e))}), error = function(e) {    NULL    if (TRUE) {        try(stop(e))    }    else {        invisible()    }}, interrupt = function(e) {    NULL    if (TRUE) {        e    }    else {        invisible()    }})
   An irrecoverable exception occurred. R is aborting now ...
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] jiggunjer commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "jiggunjer (via GitHub)" <gi...@apache.org>.
jiggunjer commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1435534100

   I encountered segfault when writing tables using pythons parquetwriter, in particular compression=gzip would segfault every time. I noticed that making the code non-threaded fixed the issue, so perhaps the compression types are using too many resources?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458215267

   @egillax Is the package you're testing available publicly? It would help me a *lot* to fix this if I can reproduce locally.
   
   @PMassicotte I don't recall if we have instructions but it looks like you're on Ubuntu so you could probably do `R -d gdb` and then type `run`, which should drop you into an R prompt. If your session crashes, you should be able to type `bt` (for "backtrace"). If you don't get there, don't worry about it! The most helpful thing for fixing this will be the ability for me or Nic to reproduce this locally...even a hint (like...what are the column types of the table you're trying to write?)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] tdhock commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "tdhock (via GitHub)" <gi...@apache.org>.
tdhock commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1540653578

   write_parquet example works,
   ```
   (base) tdhock@maude-MacBookPro:~/projects/max-generalized-auc(master*)$ R --vanilla -e 'example("write_parquet",package="arrow")'
   
   R version 4.3.0 (2023-04-21) -- "Already Tomorrow"
   Copyright (C) 2023 The R Foundation for Statistical Computing
   Platform: x86_64-pc-linux-gnu (64-bit)
   
   R is free software and comes with ABSOLUTELY NO WARRANTY.
   You are welcome to redistribute it under certain conditions.
   Type 'license()' or 'licence()' for distribution details.
   
     Natural language support but running in an English locale
   
   R is a collaborative project with many contributors.
   Type 'contributors()' for more information and
   'citation()' on how to cite R or R packages in publications.
   
   Type 'demo()' for some demos, 'help()' for on-line help, or
   'help.start()' for an HTML browser interface to help.
   Type 'q()' to quit R.
   
   > example("write_parquet",package="arrow")
   
   Attaching package: ‘arrow’
   
   The following object is masked from ‘package:utils’:
   
       timestamp
   
   
   wrt_pr> ## Don't show: 
   wrt_pr> if (arrow_with_parquet()) (if (getRversion() >= "3.4") withAutoprint else force)({ # examplesIf
   wrt_pr+ ## End(Don't show)
   wrt_pr+ tf1 <- tempfile(fileext = ".parquet")
   wrt_pr+ write_parquet(data.frame(x = 1:5), tf1)
   wrt_pr+ 
   wrt_pr+ # using compression
   wrt_pr+ if (codec_is_available("gzip")) {
   wrt_pr+   tf2 <- tempfile(fileext = ".gz.parquet")
   wrt_pr+   write_parquet(data.frame(x = 1:5), tf2, compression = "gzip", compression_level = 5)
   wrt_pr+ }
   wrt_pr+ ## Don't show: 
   wrt_pr+ }) # examplesIf
   > tf1 <- tempfile(fileext = ".parquet")
   > write_parquet(data.frame(x = 1:5), tf1)
   > if (codec_is_available("gzip")) {
   +     tf2 <- tempfile(fileext = ".gz.parquet")
   +     write_parquet(data.frame(x = 1:5), tf2, compression = "gzip", compression_level = 5)
   + }
   
   wrt_pr> ## End(Don't show)
   wrt_pr> 
   wrt_pr> 
   wrt_pr> 
   > 
   > 
   (base) tdhock@maude-MacBookPro:~/projects/max-generalized-auc(master*)$ 
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] PMassicotte commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "PMassicotte (via GitHub)" <gi...@apache.org>.
PMassicotte commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458629674

   Nice! thank you for your rapid response!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] egillax commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "egillax (via GitHub)" <gi...@apache.org>.
egillax commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1452109899

   Just wanted to chime in here since I'm experiencing a very similar error. 
   
   It happens when I'm writing a dataframe to an arrow/feather file. Like for OP it works in arrow 10 but not 11. And is originating from the same line.
   
   Here is the top of the stacktrace running R with debugger attached:
   ```
   * thread #1, name = 'R', stop reason = signal SIGSEGV: invalid address (fault address: 0x18)
     * frame #0: 0x00007fff89fb0516 arrow.so`arrow::r::InferArrowType(SEXPREC*) at shared_ptr_base.h:1522:7
       frame #1: 0x00007fff89fab73e arrow.so`arrow::r::InferSchemaFromDots(SEXPREC*, SEXPREC*, int, std::shared_ptr<arrow::Schema>&)::'lambda'(int, SEXPREC*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>)::operator()(int, SEXPREC*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>) const at table.cpp:179:62
       frame #2: 0x00007fff89fac6c1 arrow.so`arrow::r::InferSchemaFromDots(SEXPREC*, SEXPREC*, int, std::shared_ptr<arrow::Schema>&) at arrow_types.h:211:15
       frame #3: 0x00007fff89f413aa arrow.so`Table__from_dots(SEXPREC*, SEXPREC*, bool) at r_to_arrow.cpp:1461:44
       frame #4: 0x00007fff89e9a690 arrow.so`_arrow_Table__from_dots at arrowExports.cpp:4321:40
   ```
   
   It happens during unit testing in my package so I can reproduce it as much as I want locally, and happens as well running github actions on ubuntu, windows and macOS. But unfortunately have not been able to create a minimum reproducible example. Just calling the function separately that gives the issue with the same inputs does not give the error. So it seems it is dependent on something that happens earlier during my unit testing.  
   
   I've tried turning of thread using ```options(arrow.use_threads = FALSE)``` but that doesn't solve the issue.
   
   Attached is my sessionInfo and arrowInfo. If you have any ideas on how to debug further please let me know. 
   
   <details>
     <summary>SessionInfo</summary>
     R version 4.2.2 (2022-10-31)
   Platform: x86_64-pc-linux-gnu (64-bit)
   Running under: Ubuntu 22.10
   
   Matrix products: default
   BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.10.1
   LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.1
   
   locale:
    [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=nl_NL.UTF-8        LC_COLLATE=en_US.UTF-8    
    [5] LC_MONETARY=nl_NL.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=nl_NL.UTF-8       LC_NAME=C                 
    [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=nl_NL.UTF-8 LC_IDENTIFICATION=C       
   
   attached base packages:
   [1] stats     graphics  grDevices utils     datasets  methods   base     
   
   other attached packages:
   [1] arrow_11.0.0.2
   
   loaded via a namespace (and not attached):
    [1] tidyselect_1.2.0 bit_4.0.4        compiler_4.2.2   magrittr_2.0.3   assertthat_0.2.1 R6_2.5.1        
    [7] cli_3.6.0        tools_4.2.2      glue_1.6.2       rstudioapi_0.14  bit64_4.0.5      vctrs_0.5.1     
   [13] lifecycle_1.0.3  rlang_1.0.6      purrr_1.0.1  
    
   </details>
   
   <details>
    <summary>arrowInfo</summary>
    Arrow package version: 11.0.0.2
   
   Capabilities:
                  
   dataset    TRUE
   substrait FALSE
   parquet    TRUE
   json       TRUE
   s3         TRUE
   gcs        TRUE
   utf8proc   TRUE
   re2        TRUE
   snappy     TRUE
   gzip       TRUE
   brotli     TRUE
   zstd       TRUE
   lz4        TRUE
   lz4_frame  TRUE
   lzo       FALSE
   bz2        TRUE
   jemalloc   TRUE
   mimalloc   TRUE
   
   Memory:
                     
   Allocator jemalloc
   Current    0 bytes
   Max        0 bytes
   
   Runtime:
                           
   SIMD Level          avx2
   Detected SIMD Level avx2
   
   Build:
                              
   C++ Library Version  11.0.0
   C++ Compiler            GNU
   C++ Compiler Version 11.3.0
    </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] PMassicotte commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "PMassicotte (via GitHub)" <gi...@apache.org>.
PMassicotte commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458072361

   @thisisnic Sure. How can I do that? Is there documentation showing how to attach the output of the debugger?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458469340

   If I'm not mistaken, I believe you're even using a package I wrote to read the NetCDFs!
   
   I don't see any odd column types here but there are certainly a lot of columns and that may be helpful to help us make a reproducer.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458223111

   Based on @jiggunjer's observation, you all could also try setting:
   
   ```
   arrow:::SetEnableSignalStopSource(FALSE)
   ```
   
   (We added some cancellation features in 11.0.0 and that is a way to turn them off)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] PMassicotte commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "PMassicotte (via GitHub)" <gi...@apache.org>.
PMassicotte commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458351343

   @paleolimbot 
   
   Here is what I am getting:
   
   ```
   > source("R/001c_bioargo_chla_dark_correction.R")
   [New Thread 0x7ffeca9ff640 (LWP 2054027)]
   [New Thread 0x7ffeca1fe640 (LWP 2054028)]
   [New Thread 0x7ffec99fd640 (LWP 2054029)]
   [New Thread 0x7ffec91fc640 (LWP 2054030)]
   [New Thread 0x7ffec89fb640 (LWP 2054031)]
   [New Thread 0x7ffeb3fff640 (LWP 2054032)]
   [New Thread 0x7ffeab7fe640 (LWP 2054033)]
   [New Thread 0x7ffeb37fe640 (LWP 2054034)]
   [New Thread 0x7ffeb2ffd640 (LWP 2054035)]
   [New Thread 0x7ffeb27fc640 (LWP 2054036)]
   [New Thread 0x7ffeb1ffb640 (LWP 2054037)]
   [New Thread 0x7ffeb17fa640 (LWP 2054038)]
   [New Thread 0x7ffeb0ff9640 (LWP 2054039)]
   [New Thread 0x7ffeabfff640 (LWP 2054040)]
   [New Thread 0x7ffeaaffd640 (LWP 2054041)]
   [New Thread 0x7ffeaa7fc640 (LWP 2054042)]
   [New Thread 0x7ffea9ffb640 (LWP 2054043)]
   [New Thread 0x7ffea8fff640 (LWP 2054044)]
   [New Thread 0x7ffe73fff640 (LWP 2054045)]
   [New Thread 0x7ffe7bfff640 (LWP 2054046)]
   [New Thread 0x7ffe7b7fe640 (LWP 2054047)]
   [New Thread 0x7ffe7affd640 (LWP 2054048)]
   [New Thread 0x7ffe7a7fc640 (LWP 2054049)]
   [New Thread 0x7ffe79ffb640 (LWP 2054050)]
   [New Thread 0x7ffe797fa640 (LWP 2054051)]
   [New Thread 0x7ffe78ff9640 (LWP 2054052)]
   [New Thread 0x7ffe737fe640 (LWP 2054053)]
   [New Thread 0x7ffe72ffd640 (LWP 2054054)]
   [New Thread 0x7ffe727fc640 (LWP 2054055)]
   [New Thread 0x7ffe71ffb640 (LWP 2054056)]
   [New Thread 0x7ffe717fa640 (LWP 2054057)]
   [New Thread 0x7ffe70ff9640 (LWP 2054058)]
   [New Thread 0x7ffe3bfff640 (LWP 2054059)]
   [New Thread 0x7ffe3b7fe640 (LWP 2054060)]
   [New Thread 0x7ffe3aafd640 (LWP 2054061)]
   Joining with `by = join_by(takuse, date_time, n_prof)`
   Joining with `by = join_by(takuse, date_time, n_prof)`
   [New Thread 0x7ffecb66f640 (LWP 2054218)]
   [Thread 0x7ffecb66f640 (LWP 2054218) exited]
   [New Thread 0x7ffecb66f640 (LWP 2054219)]
   [New Thread 0x7ffdfb1ff640 (LWP 2054220)]
   [Thread 0x7ffecb66f640 (LWP 2054219) exited]
   [New Thread 0x7ffecb66f640 (LWP 2054221)]
   [New Thread 0x7ffdfa9fe640 (LWP 2054222)]
   [Thread 0x7ffdfb1ff640 (LWP 2054220) exited]
   [Thread 0x7ffecb66f640 (LWP 2054221) exited]
   [Thread 0x7ffdfa9fe640 (LWP 2054222) exited]
   [New Thread 0x7ffdfa9fe640 (LWP 2054223)]
   [New Thread 0x7ffecb66f640 (LWP 2054224)]
   [Thread 0x7ffdfa9fe640 (LWP 2054223) exited]
   [New Thread 0x7ffdfa9fe640 (LWP 2054225)]
   [New Thread 0x7ffdfb1ff640 (LWP 2054226)]
   [Thread 0x7ffecb66f640 (LWP 2054224) exited]
   [Thread 0x7ffdfa9fe640 (LWP 2054225) exited]
   [New Thread 0x7ffdfa9fe640 (LWP 2054249)]
   [New Thread 0x7ffecb66f640 (LWP 2054250)]
   [Thread 0x7ffdfb1ff640 (LWP 2054226) exited]
   [Thread 0x7ffdfa9fe640 (LWP 2054249) exited]
   [New Thread 0x7ffdfa9fe640 (LWP 2054251)]
   [New Thread 0x7ffdfb1ff640 (LWP 2054252)]
   [Thread 0x7ffecb66f640 (LWP 2054250) exited]
   [Thread 0x7ffdfa9fe640 (LWP 2054251) exited]
   [Thread 0x7ffdfb1ff640 (LWP 2054252) exited]
   
   Thread 1 "R" received signal SIGSEGV, Segmentation fault.
   0x00007ffed2b6cffd in arrow::r::InferArrowType(SEXPREC*) () from /home/filoche/R/x86_64-pc-linux-gnu-library/4.2/arrow/libs/arrow.so
   (gdb) 
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] PMassicotte commented on issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "PMassicotte (via GitHub)" <gi...@apache.org>.
PMassicotte commented on issue #34211:
URL: https://github.com/apache/arrow/issues/34211#issuecomment-1458383060

   Ah, cool to meet a fellow bioargo colleague :)
   
   ```r
   
   glimpse(bioargo_dark_corrected)
   
   Rows: 326,727
   Columns: 111
   $ filename                          <chr> "4902602_Sprof.nc", "4…
   $ floatname                         <chr> "4902602", "4902602", …
   $ takuse                            <chr> "takuse002b", "takuse0…
   $ date_time                         <dttm> 2021-10-26 14:23:06, …
   $ juld                              <dbl> 26231.6, 26231.6, 2623…
   $ juld_qc                           <chr> "1", "1", "1", "1", "1…
   $ juld_location                     <dbl> 26231.62, 26231.62, 26…
   $ pres                              <dbl> 0.00, 0.10, 0.13, 0.28…
   $ pres_qc                           <chr> "1", "1", "1", "1", "1…
   $ pres_adjusted                     <dbl> NA, NA, NA, NA, -0.050…
   $ pres_adjusted_qc                  <chr> " ", " ", " ", " ", "1…
   $ pres_adjusted_error               <dbl> NA, NA, NA, NA, NA, NA…
   $ temp                              <dbl> -0.5600167, -0.5598500…
   $ temp_qc                           <chr> "3", "3", "3", "3", "3…
   $ temp_d_pres                       <dbl> -0.05, 0.03, 0.00, 0.0…
   $ temp_adjusted                     <dbl> -0.5680000, -0.5680000…
   $ temp_adjusted_qc                  <chr> "8", "1", "8", "8", "1…
   $ temp_adjusted_error               <dbl> NA, NA, NA, NA, NA, NA…
   $ psal                              <dbl> 31.58545, 31.58433, 31…
   $ psal_qc                           <chr> "3", "3", "3", "3", "3…
   $ psal_d_pres                       <dbl> -0.05, 0.03, 0.00, 0.0…
   $ psal_adjusted                     <dbl> 31.58900, 31.58900, 31…
   $ psal_adjusted_qc                  <chr> "8", "1", "8", "8", "1…
   $ psal_adjusted_error               <dbl> NA, NA, NA, NA, NA, NA…
   $ doxy                              <dbl> NA, NA, NA, NA, NA, NA…
   $ doxy_qc                           <chr> " ", " ", " ", " ", " …
   $ doxy_d_pres                       <dbl> NA, NA, NA, NA, NA, NA…
   $ doxy_adjusted                     <dbl> NA, NA, NA, NA, NA, NA…
   $ doxy_adjusted_qc                  <chr> " ", " ", " ", " ", " …
   $ doxy_adjusted_error               <dbl> NA, NA, NA, NA, NA, NA…
   $ down_irradiance380                <dbl> 0.004928912, 0.0049844…
   $ down_irradiance380_qc             <chr> "1", "1", " ", " ", "8…
   $ down_irradiance380_d_pres         <dbl> 0.02, 0.02, NA, NA, -0…
   $ down_irradiance380_adjusted       <dbl> NA, NA, NA, NA, NA, NA…
   $ down_irradiance380_adjusted_qc    <chr> " ", " ", " ", " ", " …
   $ down_irradiance380_adjusted_error <dbl> NA, NA, NA, NA, NA, NA…
   $ down_irradiance412                <dbl> 0.007823820, 0.0079019…
   $ down_irradiance412_qc             <chr> "1", "1", " ", " ", "8…
   $ down_irradiance412_d_pres         <dbl> 0.02, 0.02, NA, NA, -0…
   $ down_irradiance412_adjusted       <dbl> NA, NA, NA, NA, NA, NA…
   $ down_irradiance412_adjusted_qc    <chr> " ", " ", " ", " ", " …
   $ down_irradiance412_adjusted_error <dbl> NA, NA, NA, NA, NA, NA…
   $ down_irradiance490                <dbl> 0.007710332, 0.0077336…
   $ down_irradiance490_qc             <chr> "1", "1", " ", " ", "8…
   $ down_irradiance490_d_pres         <dbl> 0.02, 0.02, NA, NA, -0…
   $ down_irradiance490_adjusted       <dbl> NA, NA, NA, NA, NA, NA…
   $ down_irradiance490_adjusted_qc    <chr> " ", " ", " ", " ", " …
   $ down_irradiance490_adjusted_error <dbl> NA, NA, NA, NA, NA, NA…
   $ downwelling_par                   <dbl> 7.388872, 7.413715, NA…
   $ downwelling_par_qc                <chr> "1", "1", " ", " ", "8…
   $ downwelling_par_d_pres            <dbl> 0.02, 0.02, NA, NA, -0…
   $ downwelling_par_adjusted          <dbl> NA, NA, NA, NA, NA, NA…
   $ downwelling_par_adjusted_qc       <chr> " ", " ", " ", " ", " …
   $ downwelling_par_adjusted_error    <dbl> NA, NA, NA, NA, NA, NA…
   $ chla                              <dbl> NA, 0.489100, NA, NA, …
   $ chla_qc                           <chr> " ", "3", " ", " ", "3…
   $ chla_d_pres                       <dbl> NA, 0.0, NA, NA, 0.0, …
   $ chla_adjusted                     <dbl> NA, 0.2993, NA, NA, 0.…
   $ chla_adjusted_qc                  <chr> " ", "5", " ", " ", "5…
   $ chla_adjusted_error               <dbl> NA, NA, NA, NA, NA, NA…
   $ bbp700                            <dbl> NA, 0.0007781871, NA, …
   $ bbp700_qc                         <chr> " ", "2", " ", " ", "2…
   $ bbp700_d_pres                     <dbl> NA, 0.0, NA, NA, 0.0, …
   $ bbp700_adjusted                   <dbl> NA, NA, NA, NA, NA, NA…
   $ bbp700_adjusted_qc                <chr> " ", " ", " ", " ", " …
   $ bbp700_adjusted_error             <dbl> NA, NA, NA, NA, NA, NA…
   $ cdom                              <dbl> NA, 0.7080, NA, NA, 0.…
   $ cdom_qc                           <chr> " ", "0", " ", " ", "0…
   $ cdom_d_pres                       <dbl> NA, 0.0, NA, NA, 0.0, …
   $ cdom_adjusted                     <dbl> NA, NA, NA, NA, NA, NA…
   $ cdom_adjusted_qc                  <chr> " ", " ", " ", " ", " …
   $ cdom_adjusted_error               <dbl> NA, NA, NA, NA, NA, NA…
   $ nitrate                           <dbl> NA, NA, NA, NA, NA, NA…
   $ nitrate_qc                        <chr> " ", " ", " ", " ", " …
   $ nitrate_d_pres                    <dbl> NA, NA, NA, NA, NA, NA…
   $ nitrate_adjusted                  <dbl> NA, NA, NA, NA, NA, NA…
   $ nitrate_adjusted_qc               <chr> " ", " ", " ", " ", " …
   $ nitrate_adjusted_error            <dbl> NA, NA, NA, NA, NA, NA…
   $ n_levels                          <int> 4, 5, 6, 7, 8, 9, 10, …
   $ n_prof                            <int> 1, 1, 1, 1, 1, 1, 1, 1…
   $ cycle_number                      <int> 1, 1, 1, 1, 1, 1, 1, 1…
   $ direction                         <chr> "A", "A", "A", "A", "A…
   $ latitude                          <dbl> 72.71641, 72.71641, 72…
   $ longitude                         <dbl> -66.70568, -66.70568, …
   $ position_qc                       <chr> "1", "1", "1", "1", "1…
   $ config_mission_number             <int> 1, 1, 1, 1, 1, 1, 1, 1…
   $ profile_pres_qc                   <chr> "A", "A", "A", "A", "A…
   $ profile_temp_qc                   <chr> "B", "B", "B", "B", "B…
   $ profile_psal_qc                   <chr> "B", "B", "B", "B", "B…
   $ profile_doxy_qc                   <chr> "B", "B", "B", "B", "B…
   $ profile_down_irradiance380_qc     <chr> "A", "A", "A", "A", "A…
   $ profile_down_irradiance412_qc     <chr> "A", "A", "A", "A", "A…
   $ profile_down_irradiance490_qc     <chr> "A", "A", "A", "A", "A…
   $ profile_downwelling_par_qc        <chr> "A", "A", "A", "A", "A…
   $ profile_chla_qc                   <chr> "B", "B", "B", "B", "B…
   $ profile_bbp700_qc                 <chr> "A", "A", "A", "A", "A…
   $ profile_cdom_qc                   <chr> " ", " ", " ", " ", " …
   $ profile_nitrate_qc                <chr> "A", "A", "A", "A", "A…
   $ position                          <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_pres                      <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_temp                      <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_psal                      <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_doxy                      <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_down_irradiance380        <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_down_irradiance412        <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_down_irradiance490        <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_downwelling_par           <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_chla                      <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_bbp700                    <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_cdom                      <lgl> NA, NA, NA, NA, NA, NA…
   $ profile_nitrate                   <lgl> NA, NA, NA, NA, NA, NA…
   ```
   
   ```r
   r$> str(bioargo_dark_corrected[integer(0), ])
   tibble [0 × 111] (S3: tbl_df/tbl/data.frame)
    $ filename                         : chr(0) 
    $ floatname                        : chr(0) 
    $ takuse                           : Named chr(0) 
     ..- attr(*, "names")= chr(0) 
    $ date_time                        : 'POSIXct' num(0) 
    - attr(*, "tzone")= chr "UTC"
    $ juld                             : num(0) 
    $ juld_qc                          : chr(0) 
    $ juld_location                    : num(0) 
    $ pres                             : num(0) 
    $ pres_qc                          : chr(0) 
    $ pres_adjusted                    : num(0) 
    $ pres_adjusted_qc                 : chr(0) 
    $ pres_adjusted_error              : num(0) 
    $ temp                             : num(0) 
    $ temp_qc                          : chr(0) 
    $ temp_d_pres                      : num(0) 
    $ temp_adjusted                    : num(0) 
    $ temp_adjusted_qc                 : chr(0) 
    $ temp_adjusted_error              : num(0) 
    $ psal                             : num(0) 
    $ psal_qc                          : chr(0) 
    $ psal_d_pres                      : num(0) 
    $ psal_adjusted                    : num(0) 
    $ psal_adjusted_qc                 : chr(0) 
    $ psal_adjusted_error              : num(0) 
    $ doxy                             : num(0) 
    $ doxy_qc                          : chr(0) 
    $ doxy_d_pres                      : num(0) 
    $ doxy_adjusted                    : num(0) 
    $ doxy_adjusted_qc                 : chr(0) 
    $ doxy_adjusted_error              : num(0) 
    $ down_irradiance380               : num(0) 
    $ down_irradiance380_qc            : chr(0) 
    $ down_irradiance380_d_pres        : num(0) 
    $ down_irradiance380_adjusted      : num(0) 
    $ down_irradiance380_adjusted_qc   : chr(0) 
    $ down_irradiance380_adjusted_error: num(0) 
    $ down_irradiance412               : num(0) 
    $ down_irradiance412_qc            : chr(0) 
    $ down_irradiance412_d_pres        : num(0) 
    $ down_irradiance412_adjusted      : num(0) 
    $ down_irradiance412_adjusted_qc   : chr(0) 
    $ down_irradiance412_adjusted_error: num(0) 
    $ down_irradiance490               : num(0) 
    $ down_irradiance490_qc            : chr(0) 
    $ down_irradiance490_d_pres        : num(0) 
    $ down_irradiance490_adjusted      : num(0) 
    $ down_irradiance490_adjusted_qc   : chr(0) 
    $ down_irradiance490_adjusted_error: num(0) 
    $ downwelling_par                  : num(0) 
    $ downwelling_par_qc               : chr(0) 
    $ downwelling_par_d_pres           : num(0) 
    $ downwelling_par_adjusted         : num(0) 
    $ downwelling_par_adjusted_qc      : chr(0) 
    $ downwelling_par_adjusted_error   : num(0) 
    $ chla                             : num(0) 
    $ chla_qc                          : chr(0) 
    $ chla_d_pres                      : num(0) 
    $ chla_adjusted                    : num(0) 
    $ chla_adjusted_qc                 : chr(0) 
    $ chla_adjusted_error              : num(0) 
    $ bbp700                           : num(0) 
    $ bbp700_qc                        : chr(0) 
    $ bbp700_d_pres                    : num(0) 
    $ bbp700_adjusted                  : num(0) 
    $ bbp700_adjusted_qc               : chr(0) 
    $ bbp700_adjusted_error            : num(0) 
    $ cdom                             : num(0) 
    $ cdom_qc                          : chr(0) 
    $ cdom_d_pres                      : num(0) 
    $ cdom_adjusted                    : num(0) 
    $ cdom_adjusted_qc                 : chr(0) 
    $ cdom_adjusted_error              : num(0) 
    $ nitrate                          : num(0) 
    $ nitrate_qc                       : chr(0) 
    $ nitrate_d_pres                   : num(0) 
    $ nitrate_adjusted                 : num(0) 
    $ nitrate_adjusted_qc              : chr(0) 
    $ nitrate_adjusted_error           : num(0) 
    $ n_levels                         : int(0) 
    $ n_prof                           : int(0) 
    $ cycle_number                     : int(0) 
    $ direction                        : chr(0) 
    $ latitude                         : num(0) 
    $ longitude                        : num(0) 
    $ position_qc                      : chr(0) 
    $ config_mission_number            : int(0) 
    $ profile_pres_qc                  : chr(0) 
    $ profile_temp_qc                  : chr(0) 
    $ profile_psal_qc                  : chr(0) 
    $ profile_doxy_qc                  : chr(0) 
    $ profile_down_irradiance380_qc    : chr(0) 
    $ profile_down_irradiance412_qc    : chr(0) 
    $ profile_down_irradiance490_qc    : chr(0) 
    $ profile_downwelling_par_qc       : chr(0) 
    $ profile_chla_qc                  : chr(0) 
    $ profile_bbp700_qc                : chr(0) 
    $ profile_cdom_qc                  : chr(0) 
    $ profile_nitrate_qc               : chr(0) 
    $ position                         : logi(0) 
     [list output truncated]
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] thisisnic closed issue #34211: [R] Segmentation fault when using write_parquet()

Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic closed issue #34211: [R] Segmentation fault when using write_parquet()
URL: https://github.com/apache/arrow/issues/34211


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org