Re: [I] [Python] Reading partial data/first block hangs on some cloud filesystems [arrow]

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on issue #43497:
URL: https://github.com/apache/arrow/issues/43497#issuecomment-2358649730

   Ok, it seems the request simply fails authenticating and then retries a number of times. You can see this by putting a limit on retry duration (5 seconds in the example below):
   ```python
   >>> import pyarrow.dataset as ds
   >>> uri = "gs://datachain-demo/laion-aesthetics-csv/laion_aesthetics_1024_33M_1.csv?retry_limit_seconds=5"
   >>> dataset = ds.dataset(uri, format="csv")
   Traceback (most recent call last):
     ...
   OSError: google::cloud::Status(UNAVAILABLE: Retry policy exhausted GetObjectMetadata: Could not create a OAuth2 access token to authenticate the request. The request was not sent, as such an access token is required to complete the request successfully. Learn more about Google Cloud authentication at https://cloud.google.com/docs/authentication. The underlying error message was: PerformWork() - CURL error [6]=Couldn't resolve host name)
   ```
   
   It's a pity that by default this would retry for some long on an authentication failure, though. Perhaps there's a way to avoid that?
   
   cc @coryan 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org