You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "krlmlr (via GitHub)" <gi...@apache.org> on 2023/06/07 09:44:44 UTC
[GitHub] [arrow-adbc] krlmlr opened a new pull request, #733: chore: Add remote
krlmlr opened a new pull request, #733:
URL: https://github.com/apache/arrow-adbc/pull/733
Some tests require a dev version of nanoarrow. This *should* enable `pak::pak()` to find it, but currently doesn't on my system.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] krlmlr commented on pull request #733: chore(r): Add remote
Posted by "krlmlr (via GitHub)" <gi...@apache.org>.
krlmlr commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1581448395
Thanks! Now:
```
R CMD build arrow-nanoarrow/r
# * checking for file ‘arrow-nanoarrow/r/DESCRIPTION’ ... OK
# * preparing ‘nanoarrow’:
# * checking DESCRIPTION meta-information ... OK
# * cleaning src
# * checking for LF line-endings in source and make files and shell scripts
# * checking for empty or unneeded directories
# * building ‘nanoarrow_0.1.0.9000.tar.gz’
#
R CMD INSTALL nanoarrow_0.1.0.9000.tar.gz
# * installing to library ‘/Users/kirill/Library/R/arm64/4.2/library’
# * installing *source* package ‘nanoarrow’ ...
# ** using staged installation
# Fetched bundled nanoarrow from https://github.com/apache/arrow-nanoarrow/tree/main/dist
# ** libs
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c altrep.c -o altrep.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c array.c -o array.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c array_stream.c -o array_stream.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c array_view.c -o array_view.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c as_array.c -o as_array.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c buffer.c -o buffer.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c convert.c -o convert.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c convert_array.c -o convert_array.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c convert_array_stream.c -o convert_array_stream.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c infer_ptype.c -o infer_ptype.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c init.c -o init.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c materialize.c -o materialize.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c nanoarrow.c -o nanoarrow.o
# ccache clang++ -std=gnu++17 -Wno-everything -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -O0 -g -Wmacro-redefined -c nanoarrow_cpp.cc -o nanoarrow_cpp.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c pointers.c -o pointers.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c schema.c -o schema.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c util.c -o util.o
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c version.c -o version.o
# ccache clang++ -std=gnu++17 -Wno-everything -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib -L/opt/homebrew/lib -o nanoarrow.so altrep.o array.o array_stream.o array_view.o as_array.o buffer.o convert.o convert_array.o convert_array_stream.o infer_ptype.o init.o materialize.o nanoarrow.o nanoarrow_cpp.o pointers.o schema.o util.o version.o -F/Library/Frameworks/R.framework/Versions/4.2-arm64 -framework R -Wl,-framework -Wl,CoreFoundation
# installing to /Users/kirill/Library/R/arm64/4.2/library/00LOCK-nanoarrow/00new/nanoarrow/libs
# ** R
# ** inst
# ** byte-compile and prepare package for lazy loading
# ** help
# *** installing help indices
# ** building package indices
# ** testing if installed package can be loaded from temporary location
# ** checking absolute paths in shared objects and dynamic libraries
# ** testing if installed package can be loaded from final location
# ** testing if installed package keeps a record of temporary installation path
# * DONE (nanoarrow)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] paleolimbot merged pull request #733: chore(r/adbcdrivermanager): Excplicitly depend on development nanoarrow
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot merged PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] krlmlr commented on pull request #733: chore(r): Add remote
Posted by "krlmlr (via GitHub)" <gi...@apache.org>.
krlmlr commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1580327974
@gaborcsardi: Why does pak 0.5.1.9000 (2023-06-02 06:46:52 UTC) fail to install from the remote with `pak::pak()` ? What's the canonical way to fix or work around?
``` r
pak::pak()
#> ℹ Loading metadata database
#> ✔ Loading metadata database ... done
#>
#> Error: ! error in pak subprocess
#> Caused by error:
#> ! Could not solve package dependencies:
#> * deps::/Users/kirill/git/arrow-adbc/r/adbcdrivermanager: Can't install dependency nanoarrow (>= 0.1.0.9000)
```
<sup>Created on 2023-06-07 with [reprex v2.0.2](https://reprex.tidyverse.org)</sup>
For reference, the relevant parts of `DESCRIPTION` :
```dcf
Imports:
nanoarrow (>= 0.1.0.9000)
Remotes:
apache/arrow-nanoarrow/r
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] krlmlr commented on pull request #733: chore(r): Add remote
Posted by "krlmlr (via GitHub)" <gi...@apache.org>.
krlmlr commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1580331765
Got it, thanks! Sorry for the noise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] krlmlr commented on pull request #733: chore(r): Add remote
Posted by "krlmlr (via GitHub)" <gi...@apache.org>.
krlmlr commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1581249853
Yeah. There is that hack to include code in the `cleanup` script, it will also run on `R CMD build` IIRC. But I agree there's no immediate need to act.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] krlmlr commented on pull request #733: chore(r): Add remote
Posted by "krlmlr (via GitHub)" <gi...@apache.org>.
krlmlr commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1581224290
How would a nanoarrow update help compilation failures in adbcpostgresql?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] paleolimbot commented on pull request #733: chore(r): Add remote
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1581259601
Ah, sorry, there was also a bug in the "bundle" workflow (which runs nightly, but had a bug and wasn't updating all the files): https://github.com/apache/arrow-nanoarrow/pull/213 . A fresh pull might fix that build problem!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] paleolimbot commented on pull request #733: chore(r): Add remote
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1581242034
I think you maybe running into a problem where if there is no `cmake` available on PATH, the nanoarrow R package will attempt to use the amalgamated header/source in `dist/`. Since I've been making some rather large changes to that recently, it may be that the files in `dist/` are out of date with the R package sources (I think they currently update once a week). That system is also obviously not ideal but there is also just one of me.
nanoarrow 0.2 will be released in about two weeks...it may possibly be easier to wait for that release and then remove the test skip + update the version constraint?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] krlmlr commented on pull request #733: chore(r): Add remote
Posted by "krlmlr (via GitHub)" <gi...@apache.org>.
krlmlr commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1581240014
To replicate:
```sh
R CMD build arrow-nanoarrow/r
R CMD INSTALL nanoarrow_0.1.0.9000.tar.gz
# ...
# ** using staged installation
# Fetched bundled nanoarrow from https://github.com/apache/arrow-nanoarrow/tree/main/dist
# ** libs
# ccache clang -Wno-everything -w -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG -I/opt/homebrew/include -fPIC -falign-functions=64 -Wall -g -O2 -c convert_array.c -o convert_array.o
# array.c:389:52: error: no member named 'buffer_data_type' in 'struct ArrowLayout'; did you mean 'buffer_type'?
# array_view->layout.buffer_data_type[i],
# ^~~~~~~~~~~~~~~~
# buffer_type
# ./nanoarrow.h:585:24: note: 'buffer_type' declared here
# enum ArrowBufferType buffer_type[3];
# ^
# 1 error generated.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] krlmlr commented on pull request #733: chore(r): Add remote
Posted by "krlmlr (via GitHub)" <gi...@apache.org>.
krlmlr commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1581232113
It's weird: I can install nanoarrow with `R CMD INSTALL arrow-nanoarrow/r` but not with `pak::pak("apache/arrow-nanoarrow/r")` . Will try from a clean checkout.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] paleolimbot commented on pull request #733: chore(r): Add remote
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1581245600
Yes, and `R CMD build` won't work before `Rscript bootstrap.R` (or usually in CI I just do `R CMD INSTALL` before `R CMD build`. You may be in a situation where the vendored nanoarrow is out of date (rather than missing, which will generate an error). That is the gist of the change in the forthcoming (currently unreleased) pkgbuild: https://github.com/r-lib/pkgbuild/pull/157 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #733: chore(r): Add remote
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1581186723
Hmm, do we have to update Nanoarrow?
```
** libs
using C compiler: ‘gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0’
using C++ compiler: ‘g++ (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0’
gcc -I"/opt/R/4.3.0/lib/R/include" -DNDEBUG -I/usr/local/include -fpic -g -O2 -Wall -pedantic -c altrep.c -o altrep.o
gcc -I"/opt/R/4.3.0/lib/R/include" -DNDEBUG -I/usr/local/include -fpic -g -O2 -Wall -pedantic -c array.c -o array.o
array.c: In function ‘borrow_buffer’:
array.c:389:52: error: ‘struct ArrowLayout’ has no member named ‘buffer_data_type’; did you mean ‘buffer_type’?
389 | array_view->layout.buffer_data_type[i],
| ^~~~~~~~~~~~~~~~
| buffer_type
make: *** [/opt/R/4.3.0/lib/R/etc/Makeconf:191: array.o] Error 1
ERROR: compilation failed for package ‘nanoarrow’
```
CC @paleolimbot
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #733: chore(r): Add remote
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #733:
URL: https://github.com/apache/arrow-adbc/pull/733#issuecomment-1581226423
Well, it's a nanoarrow API that it's complaining about, and it's compiling nanoarrow there, not postgresql.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org