You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "lidavidm (via GitHub)" <gi...@apache.org> on 2023/05/01 02:29:51 UTC

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

lidavidm opened a new pull request, #633:
URL: https://github.com/apache/arrow-adbc/pull/633

   Fixes #629.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] zeroshade commented on a diff in pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#discussion_r1181722401


##########
ci/scripts/python_util.sh:
##########
@@ -21,6 +21,13 @@ set -ex
 
 COMPONENTS="adbc_driver_manager adbc_driver_flightsql adbc_driver_postgresql adbc_driver_sqlite"
 
+if [[ $(uname) = "Darwin" ]] && [[ "${VCPKG_ARCH}" = "arm64" ]]; then
+    echo "Disabling adbc_driver_snowflake"
+    echo "(snowflake uses arrow v11 which can't be compiled on non-x64 platforms without noasm)"

Review Comment:
   should we add the `noasm` flag for those platforms in order to still allow it to get built?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] lidavidm commented on pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#issuecomment-1530626785

   https://github.com/apache/arrow-adbc/blob/main/c/symbols.map
   
   https://github.com/apache/arrow-adbc/blob/b965f64886026ed17f5e60f96b819af9472fe484/c/cmake_modules/AdbcDefines.cmake#L55


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] lidavidm commented on a diff in pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#discussion_r1181980323


##########
ci/scripts/python_util.sh:
##########
@@ -21,6 +21,13 @@ set -ex
 
 COMPONENTS="adbc_driver_manager adbc_driver_flightsql adbc_driver_postgresql adbc_driver_sqlite"
 
+if [[ $(uname) = "Darwin" ]] && [[ "${VCPKG_ARCH}" = "arm64" ]]; then
+    echo "Disabling adbc_driver_snowflake"
+    echo "(snowflake uses arrow v11 which can't be compiled on non-x64 platforms without noasm)"

Review Comment:
   I can try that but want to at least get CI green once here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] lidavidm commented on pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#issuecomment-1530625958

   > We can pass arbitrary options to the compiler and linker in the go compile command when we build the shared library. So if you know the flags to pass to GCC to hide the symbols I can add them to command and in theory it should work....
   
   We should pass the linker script then that the other drivers use.
   
   All the Go symbols (FlightSQLNewDatabase) + a bunch of cgo-generated trampolines are exposed. Not a big deal necessarily, but it fails the check.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] zeroshade commented on a diff in pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#discussion_r1181721235


##########
c/driver/snowflake/README.md:
##########
@@ -0,0 +1,31 @@
+<!---
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
+
+# ADBC SQLite Driver

Review Comment:
   ```suggestion
   # ADBC Snowflake Driver
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] zeroshade commented on pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#issuecomment-1529961889

   @lidavidm What symbols are getting added that we want to hide?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] zeroshade commented on pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#issuecomment-1531898284

   i'll play with the symbols map and see if I can get it working.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] lidavidm commented on pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#issuecomment-1533913071

   Not packaged for macOS/AArch64.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] zeroshade commented on pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#issuecomment-1529771794

   And for snowflake, once v12 is released I'll upgrade the snowflake lib to use it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] lidavidm commented on pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#issuecomment-1529633223

   Ah, and it looks like the snowflake driver can't be cross-compiled because it uses arrow 11, and there was a bug that was only fixed in arrow 12.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] zeroshade commented on pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#issuecomment-1531995257

   @lidavidm #640 updates the cmake config to hide the symbols for the Go libs :smile: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] lidavidm commented on pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#issuecomment-1529279427

   CC @zeroshade 
   
   Do you know off the top of your head if/how we can have CGO hide symbols? Otherwise I may disable the check for the Go-based drivers


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] zeroshade commented on pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#issuecomment-1529771105

   We can pass arbitrary options to the compiler and linker in the go compile command when we build the shared library. So if you know the flags to pass to GCC to hide the symbols I can add them to command and in theory it should work....


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] lidavidm commented on pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633#issuecomment-1533703718

   @zeroshade seems even with `noasm` it doesn't work:
   
   ```
   Error: /Users/runner/go/pkg/mod/github.com/apache/arrow/go/v11@v11.0.0/arrow/compute/internal/kernels/basic_arithmetic_noasm.go:27:9: undefined: getGoArithmeticOpFloatingSameType
   Error: /Users/runner/go/pkg/mod/github.com/apache/arrow/go/v11@v11.0.0/arrow/compute/internal/kernels/basic_arithmetic_noasm.go:35:9: undefined: getGoArithmeticFixedIntOut
   ```
   
   Guessing the upgrade will take a while? I may punt on arm64 for now


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-adbc] lidavidm merged pull request #633: feat(python/adbc_driver_snowflake): package the Snowflake driver

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm merged PR #633:
URL: https://github.com/apache/arrow-adbc/pull/633


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org