You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "raulcd (via GitHub)" <gi...@apache.org> on 2024/09/02 12:16:17 UTC

[PR] GH-43905: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

raulcd opened a new pull request, #43906:
URL: https://github.com/apache/arrow/pull/43906

   ### Rationale for this change
   
   The following PR:
   - https://github.com/apache/arrow/pull/41580
   
   Made mandatory for a test the requirement to have `PARQUET_TEST_DATA` env defined.
   
   This is missing from `python_test_emscripten.sh`
   
   ### What changes are included in this PR?
   
   Add the required env to pass the test.
   
   ### Are these changes tested?
   
   Via archery
   
   ### Are there any user-facing changes?
   
   No


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43905: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2325937985

   @github-actions crossbow submit test-conda-python-emscripten


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43905: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2325076378

   cc @kou for insight. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2326166486

   @github-actions crossbow submit test-conda-python-emscripten


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43905: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2324616710

   :warning: GitHub issue #43905 **has been automatically assigned in GitHub** to PR creator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43905: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2324616587

   @github-actions crossbow submit test-conda-python-emscripten


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2355049939

   I am marking this as ready for review just skipping the test, I can create an issue as a follow up in order to mount  `PARQUET_TEST_DATA` if someone wants to take that over. @pitrou @jorisvandenbossche are you ok skipping the test to fix CI?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2355472828

   @raulcd Can you fix this PR's title and description?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on code in PR #43906:
URL: https://github.com/apache/arrow/pull/43906#discussion_r1741835323


##########
python/pyarrow/tests/parquet/conftest.py:
##########
@@ -30,6 +31,8 @@ def datadir(base_datadir):
 
 @pytest.fixture(scope='module')
 def parquet_test_datadir():
+    if sys.platform in ('emscripten', ):

Review Comment:
   ```suggestion
       if sys.platform == 'emscripten':
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2354943277

   @github-actions crossbow submit test-conda-python-emscripten


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Skip test that requires PARQUET_TEST_DATA env on emscripten [arrow]

Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2357912584

   After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit fe39c8f5518a8b57723d457a8048cc03715bd389.
   
   There were no benchmark performance regressions. 🎉
   
   The [full Conbench report](https://github.com/apache/arrow/runs/30305908936) has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Skip test that requires PARQUET_TEST_DATA env on emscripten [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2355493812

   > @raulcd Can you fix this PR's title and description?
   
   Sorry about that, I forgot to update the PR title + desc after several iterations. Thanks for double checking @pitrou 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2354951004

   Revision: e918ffa2fbdd9472646f1170d6748b05f1a5387d
   
   Submitted crossbow builds: [ursacomputing/crossbow @ actions-295c35fb98](https://github.com/ursacomputing/crossbow/branches/all?query=actions-295c35fb98)
   
   |Task|Status|
   |----|------|
   |test-conda-python-emscripten|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-295c35fb98-github-test-conda-python-emscripten)](https://github.com/ursacomputing/crossbow/actions/runs/10899849615/job/30246176852)|


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2326126221

   For setting the env variable, could we modify `os.environ` directly in python code? (eg in `_load_pyarrow_in_runner`)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43905: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2324621603

   Revision: 3091b519c41ce62e702fbee8f4dc7692a5bc72ff
   
   Submitted crossbow builds: [ursacomputing/crossbow @ actions-d04eba703d](https://github.com/ursacomputing/crossbow/branches/all?query=actions-d04eba703d)
   
   |Task|Status|
   |----|------|
   |test-conda-python-emscripten|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-d04eba703d-github-test-conda-python-emscripten)](https://github.com/ursacomputing/crossbow/actions/runs/10666982928/job/29563590335)|


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43905: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2325429731

   I think that we can't inject Parquet test data directory path by environment variable.
   Because environment variable doesn't exist on Web browser.
   
   But... we may be able to use https://github.com/pyodide/pyodide/blob/a448c045cf5640c5050e0c3722bafc0d58659fcc/src/js/pyodide.ts#L151-L160 :
   
   ```diff
   diff --git a/python/scripts/run_emscripten_tests.py b/python/scripts/run_emscripten_tests.py
   index 1a4b4a4e05..79996b1f92 100644
   --- a/python/scripts/run_emscripten_tests.py
   +++ b/python/scripts/run_emscripten_tests.py
   @@ -19,6 +19,7 @@
    import argparse
    import contextlib
    import http.server
   +import json
    import os
    import queue
    import shutil
   @@ -153,7 +154,7 @@ class NodeDriver:
            self.execute_js(
                f"""
            const {{ loadPyodide }} = require('{dist_dir}/pyodide.js');
   -        let pyodide = await loadPyodide();
   +        let pyodide = await loadPyodide({json.dumps({"PARQUET_TEST_DATA": os.environ.get("PARQUET_TEST_DATA")})});
            """
            )
    
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2325940978

   :warning: GitHub issue #43868 **has been automatically assigned in GitHub** to PR creator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Skip test that requires PARQUET_TEST_DATA env on emscripten [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou merged PR #43906:
URL: https://github.com/apache/arrow/pull/43906


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2325943114

   Revision: 3d3708aed26eae449356f00dde909863e2c1b12a
   
   Submitted crossbow builds: [ursacomputing/crossbow @ actions-8e719208e8](https://github.com/ursacomputing/crossbow/branches/all?query=actions-8e719208e8)
   
   |Task|Status|
   |----|------|
   |test-conda-python-emscripten|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-8e719208e8-github-test-conda-python-emscripten)](https://github.com/ursacomputing/crossbow/actions/runs/10679581516/job/29599132924)|


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2326109221

   FWIW this seems to "work" (passing throught the env variable) for the Node test run ("Running emscripten tests in Node"), but then it still fails it does not find that file:
   
   ```
    =================================== FAILURES ===================================
   _____________________ test_column_chunk_key_value_metadata _____________________
   
   parquet_test_datadir = PosixPath('/arrow/cpp/submodules/parquet-testing/data')
   
       def test_column_chunk_key_value_metadata(parquet_test_datadir):
   >       metadata = pq.read_metadata(parquet_test_datadir /
                                       'column_chunk_key_value_metadata.parquet')
   
   /lib/python3.12/site-packages/pyarrow/tests/parquet/test_metadata.py:788: 
   _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
   /lib/python3.12/site-packages/pyarrow/parquet/core.py:2297: in read_metadata
       file_ctx = where = filesystem.open_input_file(where)
   pyarrow/_fs.pyx:789: in pyarrow._fs.FileSystem.open_input_file
       ???
   pyarrow/error.pxi:155: in pyarrow.lib.pyarrow_internal_check_status
       ???
   _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
   
   >   ???
   E   FileNotFoundError: [Errno 44] Failed to open local file '/arrow/cpp/submodules/parquet-testing/data/column_chunk_key_value_metadata.parquet'. Detail: [errno 44] No such file or directory
   
   pyarrow/error.pxi:92: FileNotFoundError
   ```
   
   The above part is a bit hidden in the output logs, because the second test run ("Running emscripten tests in Chrome") still gives the same error about the env variable not set. 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2326171243

   Revision: 6a5c5143315fe3ef2d1f57ee6de7b70d3307d95c
   
   Submitted crossbow builds: [ursacomputing/crossbow @ actions-e9434c5a2d](https://github.com/ursacomputing/crossbow/branches/all?query=actions-e9434c5a2d)
   
   |Task|Status|
   |----|------|
   |test-conda-python-emscripten|[![GitHub Actions](https://github.com/ursacomputing/crossbow/actions/workflows/crossbow.yml/badge.svg?branch=actions-e9434c5a2d-github-test-conda-python-emscripten)](https://github.com/ursacomputing/crossbow/actions/runs/10681207252/job/29604342097)|


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2326168084

   Pushing a test skip that works locally but happy if @joemarshall or someone else comes with a solution that does not require us to skip the test.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2326192548

   Skipping the test is certainly fine with me


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2329337002

   I've been able to have the test passing locally for the NodeDriver with this (and some other minor changes):
   ```diff
   diff --git a/python/scripts/run_emscripten_tests.py b/python/scripts/run_emscripten_tests.py
   index 53d3dd5..550e1a0 100644
   --- a/python/scripts/run_emscripten_tests.py
   +++ b/python/scripts/run_emscripten_tests.py
   @@ -19,6 +19,7 @@
    import argparse
    import contextlib
    import http.server
   +import json
    import os
    import queue
    import shutil
   @@ -153,7 +154,8 @@ class NodeDriver:
            self.execute_js(
                f"""
            const {{ loadPyodide }} = require('{dist_dir}/pyodide.js');
   -        let pyodide = await loadPyodide();
   +        let pyodide = await loadPyodide({json.dumps({"env": {"PARQUET_TEST_DATA": "/test-data"}})});
   +        pyodide.mountNodeFS("/test-data", "{os.environ.get("PARQUET_TEST_DATA")}");
            """
            )
   
   ```
   but the test still fails for the ChromeDriver, I tried a couple of things, like:
   ```diff
   @@ -203,7 +205,12 @@ class BrowserDriver:
            self.driver.set_script_timeout(100)
    
        def load_pyodide(self, dist_dir):
   -        pass
   +        self.driver.execute_async_script(
   +            f"""
   +        let pyodide = await loadPyodide({json.dumps({"env": {"PARQUET_TEST_DATA": "/test-data"}})});
   +        pyodide.mountNodeFS("/test-data", "{os.environ.get("PARQUET_TEST_DATA")}");
   +        """
   +        )
    
   ```
   but this just crashes, so I am not sure how to do the mount for the ChromeDriver.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43905: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2324682242

   ok, this does not work on `emscripten`. I have no idea how environment variables work there and / or if they work. We probably should skip this test on that environment. I am also not sure if we should just delete this test, my expectation is that if a test requires this to be set will already fail if it does not find the parquet test data files. @jorisvandenbossche should we just remove this test?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2326103870

   I was testing locally and it seems that even though the file exists at the location inside the container:
   ```
   root@4084a63715db:/# ls -lrt /arrow/cpp/submodules/parquet-testing/data/column_chunk_key_value_metadata.parquet 
   -rw-rw-r-- 1 1000 1000 400 Sep  2 11:59 /arrow/cpp/submodules/parquet-testing/data/column_chunk_key_value_metadata.parquet
   ```
   The test fails:
   ```
   _____________________ test_column_chunk_key_value_metadata _____________________
   
   parquet_test_datadir = PosixPath('/arrow/cpp/submodules/parquet-testing/data')
   
       def test_column_chunk_key_value_metadata(parquet_test_datadir):
   >       metadata = pq.read_metadata(parquet_test_datadir /
                                       'column_chunk_key_value_metadata.parquet')
   
   /lib/python3.12/site-packages/pyarrow/tests/parquet/test_metadata.py:788: 
   _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
   /lib/python3.12/site-packages/pyarrow/parquet/core.py:2297: in read_metadata
       file_ctx = where = filesystem.open_input_file(where)
   pyarrow/_fs.pyx:789: in pyarrow._fs.FileSystem.open_input_file
       ???
   pyarrow/error.pxi:155: in pyarrow.lib.pyarrow_internal_check_status
       ???
   _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
   
   >   ???
   E   FileNotFoundError: [Errno 44] Failed to open local file '/arrow/cpp/submodules/parquet-testing/data/column_chunk_key_value_metadata.parquet'. Detail: [errno 44] No such file or directory
   
   pyarrow/error.pxi:92: FileNotFoundError
   
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2326114291

   cc @joemarshall 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-43868: [CI][Python] Add required PARQUET_TEST_DATA env to python_test_emscripten.sh [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #43906:
URL: https://github.com/apache/arrow/pull/43906#issuecomment-2326194827

   This seems relevant:
   https://pyodide.org/en/stable/usage/accessing-files.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org