[GitHub] [arrow] thvasilo commented on issue #37260: [Python] Occasional error when reading metadata for Parquet files on S3 (in parallel)

["thvasilo (via GitHub)" <gi...@apache.org>]

thvasilo commented on issue #37260:
URL: https://github.com/apache/arrow/issues/37260#issuecomment-1686732073

   Hi @mapleFU , thanks for your response. Yes `HeadObject` is the name of the [operation](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html) being made (usually to check if a file exists), but error code 503 would indicate a service busy/slow down error (in the list of [S3 error code list](https://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html) these are the only cases we can get 503).
   
   I would expect a `HeadObject` operation to only fail when the file does not exist, which is not the case here.
   
   Instead of `Slow down` or busy, we're getting `Unknown` back, so I'm wondering if https://github.com/apache/arrow/blob/94bd0d27326e8f3bfc0f612fcf604617402888ec/python/pyarrow/error.pxi#L115 is somehow losing the original error message? Is there a way to get the full original response (if any exists) like a fully verbose mode for PyArrow?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org