[GitHub] [arrow] haixuanTao opened a new issue, #34994: Creating first pyarrow is slow.

["haixuanTao (via GitHub)" <gi...@apache.org>]

haixuanTao opened a new issue, #34994:
URL: https://github.com/apache/arrow/issues/34994

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   On my machine creating the first arrow array is unexpectedly slow.
   
   ## To reproduce
   
   ```bash
   python3 -c "import pyarrow as pa; import time; t_send = time.perf_counter(); b = pa.array([200]); print(f'send time={(time.perf_counter() - t_send) * 1_000_000}')"
   ```
   result (in ns):
   ```bash
   send time=188282.6280016161 
   ```
   
   This is equal to 200ms. 
   
   While:
   
   ```bash
   python3 -c "import pyarrow as pa; import time; pa.array([200]); t_send = time.perf_counter(); b = pa.array([200]); print(f'send time={(time.perf_counter() - t_send) * 1_000_000}')"
   ``` 
   result (in ns):
   ```bash
   send time=19.691000488819554
   ```
   
   ## Expected behaviour
   
   Not sure why this warmup is required but can we have this warmup within the `__init__` of arrow.
   
   ## Version
   
   ```
   Name: pyarrow
   Version: 11.0.0
   ```
   
   ```
   Python 3.10.8
   ```
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Python] Creating first `pyarrow.array` is slow. [arrow]

["thisisnic (via GitHub)" <gi...@apache.org>]

thisisnic commented on issue #34994:
URL: https://github.com/apache/arrow/issues/34994#issuecomment-6335309286

   @AlenkaF I'm going through old `Type: bug` issues. This reports that the first `pa.array([200])` call takes about 200 ms on pyarrow 11 while later calls take microseconds, which is the one-off lazy initialisation cost (type registries, numpy/pandas shims) and moving it into `import pyarrow` would only shift it; nobody has commented since April 2023. Are you happy for me to close this as not a bug?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org