You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "charlesbluca (via GitHub)" <gi...@apache.org> on 2023/04/27 17:14:05 UTC

[GitHub] [arrow-datafusion-python] charlesbluca opened a new pull request, #350: First pass at getting architectured builds working

charlesbluca opened a new pull request, #350:
URL: https://github.com/apache/arrow-datafusion-python/pull/350

    # Rationale for this change
   <!--
    Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.  
   -->
   
   When attempting to import the generated conda nightlies on a non-Linux OS, we get an error message looking something like:
   
   ```python
   >>> import datafusion
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "/Users/charlesb/mambaforge/envs/test-datafustion-packages/lib/python3.10/site-packages/datafusion/__init__.py", line 28, in <module>
       from ._internal import (
   ImportError: dlopen(/Users/charlesb/mambaforge/envs/test-datafustion-packages/lib/python3.10/site-packages/datafusion/_internal.abi3.so, 0x0002): tried: '/Users/charlesb/mambaforge/envs/test-datafustion-packages/lib/python3.10/site-packages/datafusion/_internal.abi3.so' (not a mach-o file), '/System/Volumes/Preboot/Cryptexes/OS/Users/charlesb/mambaforge/envs/test-datafustion-packages/lib/python3.10/site-packages/datafusion/_internal.abi3.so' (no such file), '/Users/charlesb/mambaforge/envs/test-datafustion-packages/lib/python3.10/site-packages/datafusion/_internal.abi3.so' (not a mach-o file)
   ```
   
   I'm fairly sure this is due to the package being built as `noarch` right now, and I think can be resolved by specifying an architecture-dependent Rust compiler at build time (though probably won't be able to get full verification on this until we run against conda-forge's CI if we intend to publish there).
   
   # What changes are included in this PR?
   <!--
   There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
   -->
   
   Modifications to the conda nightly recipe to enable architectured builds of the nightly packages, which should hopefully unblock import errors on other operating systems.
   
   Might also push some temporary changes to roughly verify that things build correctly on Windows/macOS, though was planning to follow dask-sql's practice of only testing builds against Linux.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] First pass at getting architectured builds working [arrow-datafusion-python]

Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove merged PR #350:
URL: https://github.com/apache/arrow-datafusion-python/pull/350


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion-python] charlesbluca commented on a diff in pull request #350: First pass at getting architectured builds working

Posted by "charlesbluca (via GitHub)" <gi...@apache.org>.
charlesbluca commented on code in PR #350:
URL: https://github.com/apache/arrow-datafusion-python/pull/350#discussion_r1183904741


##########
conda/recipes/meta.yaml:
##########
@@ -15,7 +15,7 @@
 # specific language governing permissions and limitations
 # under the License.
 
-{% set name = "datafusion" %}
+{% set name = "datafusion-python" %}

Review Comment:
   Noticed that we're already publishing arrow-datafusion to conda-forge under the name `datafusion`:
   
   - https://anaconda.org/conda-forge/datafusion
   
   This shouldn't have any impact on the name of the package that actually gets imported, just prevents collision with the other package; does this name seem okay? cc @andygrove 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion-python] charlesbluca commented on a diff in pull request #350: First pass at getting architectured builds working

Posted by "charlesbluca (via GitHub)" <gi...@apache.org>.
charlesbluca commented on code in PR #350:
URL: https://github.com/apache/arrow-datafusion-python/pull/350#discussion_r1179652664


##########
conda/recipes/conda_build_config.yaml:
##########
@@ -0,0 +1,21 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+python:
+  - 3.8
+  - 3.9
+  - 3.10

Review Comment:
   Defaulted to the python versions dask-sql builds against, but can swap this to whatever build matrix makes most sense here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org