Re: [I] [Parquet][Python][Dataset] Mixed FileInfo and path sources can crash Python in dataset() [arrow]

["Mohit-25-tech (via GitHub)" <gi...@apache.org>]

Mohit-25-tech commented on issue #53033:
URL: https://github.com/apache/arrow/issues/53033#issuecomment-6101496272

   Hi maintainers,
   
   I've submitted a PR addressing this issue.
   
   **Root cause:** The dataset construction logic handles mixed `FileInfo` and path inputs based on the first element's type. This can pass a mixed list into the Cython factory, where an unchecked `FileInfo` cast causes a segmentation fault.
   
   **Fix implemented:**
   - Normalize path entries in mixed lists to `FileInfo` objects.
   - Preserve existing `FileInfo` metadata, source ordering, and duplicates.
   - Keep homogeneous input behavior unchanged.
   - Add defensive Cython type validation to prevent unsafe casts.
   - Add regression tests covering mixed inputs, tuples, duplicates, subtree filesystems, and invalid paths.
   
   **Verification:**
   - Independently reproduced the original segmentation fault using PyArrow 26.0.0 on Kaggle/Linux.
   - Verified that all four input combinations work using the patched Python wrapper with PyArrow 26.0.0.
   - 25 focused local checks passed.
   - Flake8, Python syntax, autopep8, and `git diff --check` passed.
   
   The modified Cython extension has not yet been compiled locally, so full integration validation remains pending in CI.
   
   **PR:** #53034
   
   I'd appreciate a review of the implementation and would be happy to make any necessary adjustments.
   
   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org