(arrow) branch main updated: GH-38844: [C++] S3FileSystem export s3 sdk config "use_virtual_addressing" to arrow::fs::S3Options (#38858)

[ap...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

apitrou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/main by this push:
     new 501b291abd GH-38844: [C++] S3FileSystem export s3 sdk config "use_virtual_addressing" to arrow::fs::S3Options (#38858)
501b291abd is described below

commit 501b291abd4eeec06083504d7d89f52a6092187a
Author: jiawei liang <72...@users.noreply.github.com>
AuthorDate: Wed Dec 6 01:20:56 2023 +0800

    GH-38844: [C++] S3FileSystem export s3 sdk config "use_virtual_addressing" to arrow::fs::S3Options (#38858)
    
    
    
    ### Rationale for this change
    See #38844 . we should export s3 sdk config "use_virtual_addressing" to arrow::fs::S3Options so that user can decide whether to use virtual hosted by themselves.
    
    ### Are these changes tested?
    I tested it with my own OSS(aliyun object store service) bucket.As the it is a S3 compatiable service, I think
     these changes are covered by existing tests)
    
    ### Are there any user-facing changes?
    
    Closes: #38844
    
    * Closes: #38844
    
    Lead-authored-by: ljw <we...@gmail.com>
    Co-authored-by: Antoine Pitrou <pi...@free.fr>
    Co-authored-by: jiawei liang <72...@users.noreply.github.com>
    Signed-off-by: Antoine Pitrou <an...@python.org>
---
 cpp/src/arrow/filesystem/s3fs.cc | 3 ++-
 cpp/src/arrow/filesystem/s3fs.h  | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/cpp/src/arrow/filesystem/s3fs.cc b/cpp/src/arrow/filesystem/s3fs.cc
index 2cc907a63c..511448cb2f 100644
--- a/cpp/src/arrow/filesystem/s3fs.cc
+++ b/cpp/src/arrow/filesystem/s3fs.cc
@@ -951,7 +951,8 @@ class ClientBuilder {
       client_config_.caPath = ToAwsString(internal::global_options.tls_ca_dir_path);
     }
 
-    const bool use_virtual_addressing = options_.endpoint_override.empty();
+    const bool use_virtual_addressing =
+        options_.endpoint_override.empty() || options_.force_virtual_addressing;
 
     // Set proxy options if provided
     if (!options_.proxy_options.scheme.empty()) {
diff --git a/cpp/src/arrow/filesystem/s3fs.h b/cpp/src/arrow/filesystem/s3fs.h
index 90333e05e7..13a0abde32 100644
--- a/cpp/src/arrow/filesystem/s3fs.h
+++ b/cpp/src/arrow/filesystem/s3fs.h
@@ -144,6 +144,14 @@ struct ARROW_EXPORT S3Options {
   /// Type of credentials being used. Set along with credentials_provider.
   S3CredentialsKind credentials_kind = S3CredentialsKind::Default;
 
+  /// Whether to use virtual addressing of buckets
+  ///
+  /// If true, then virtual addressing is always enabled.
+  /// If false, then virtual addressing is only enabled if `endpoint_override` is empty.
+  ///
+  /// This can be used for non-AWS backends that only support virtual hosted-style access.
+  bool force_virtual_addressing = false;
+
   /// Whether OutputStream writes will be issued in the background, without blocking.
   bool background_writes = true;