[jira] [Updated] (ARROW-17045) [C++] Reject trailing slashes on file path

["Will Jones (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-17045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Will Jones updated ARROW-17045:
-------------------------------
    Summary: [C++] Reject trailing slashes on file path  (was: [C++] GCS doesn't drop ending slash for files)

> [C++] Reject trailing slashes on file path
> ------------------------------------------
>
>                 Key: ARROW-17045
>                 URL: https://issues.apache.org/jira/browse/ARROW-17045
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++
>    Affects Versions: 8.0.0
>            Reporter: Will Jones
>            Assignee: Will Jones
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: 9.0.0
>
>          Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> We had several different behaviors when passing in file paths with trailing slashes: LocalFileSystem would return IOError, S3 would trim off the trailing slash, and GCS would keep the trailing slash as part of the file name (later creating confusion as the file would be labelled a "directory" in list calls). This PR moves them all to the behavior of LocalFileSystem: return IOError.
> The R filesystem bindings relied on the behavior provided by S3, so they are now modified to trim the trailing slash before passing down to C++.
> Here is an example of the differences in behavior between S3 and GCS:
> {code:python}
> import pyarrow.fs
> from pyarrow.fs import FileSelector
> from datetime import timedelta
> gcs = pyarrow.fs.GcsFileSystem(
>     endpoint_override="localhost:9001",
>     scheme="http",
>     anonymous=True,
>     retry_time_limit=timedelta(seconds=1),
> )
> gcs.create_dir("py_test")
> # Writing to test.txt with and without slash produces a file and a directory!?
> with gcs.open_output_stream("py_test/test.txt") as out_stream:
>     out_stream.write(b"Hello world!")
> with gcs.open_output_stream("py_test/test.txt/") as out_stream:
>     out_stream.write(b"Hello world!")
> gcs.get_file_info(FileSelector("py_test"))
> # [<FileInfo for 'py_test/test.txt': type=FileType.File, size=12>, <FileInfo for 'py_test/test.txt': type=FileType.Directory>]
> s3 = pyarrow.fs.S3FileSystem(
>     access_key="minioadmin",
>     secret_key="minioadmin",
>     scheme="http",
>     endpoint_override="localhost:9000",
>     allow_bucket_creation=True,
>     allow_bucket_deletion=True,
> )
> s3.create_dir("py-test")
> # Writing to test.txt with and without slash writes to same file
> with s3.open_output_stream("py-test/test.txt") as out_stream:
>     out_stream.write(b"Hello world!")
> with s3.open_output_stream("py-test/test.txt/") as out_stream:
>     out_stream.write(b"Hello world!")
> s3.get_file_info(FileSelector("py-test"))
> # [<FileInfo for 'py-test/test.txt': type=FileType.File, size=12>]
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)