[jira] [Commented] (DRILL-7289) fs.s3a.path.style.access does not seem to work

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-7289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16871069#comment-16871069 ] 

Steve Loughran commented on DRILL-7289:
---------------------------------------

it works on the S3A connector since hadoop-2.8 and HADOOP-12963; broadly tested as its the default for enterprise S3 stores which don't work with DNS.

if you have problems then either the config isn't complete or you are using an out of date version

Recommend

* download Hadoop 3.2 and install
* try the config in a core-site there, then hadoop fs- ls command
* or even better: cloudstore diagnostics https://github.com/steveloughran/cloudstore



> fs.s3a.path.style.access does not seem to work
> ----------------------------------------------
>
>                 Key: DRILL-7289
>                 URL: https://issues.apache.org/jira/browse/DRILL-7289
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Functions - Drill
>    Affects Versions: 1.16.0
>         Environment: Running on Kubernetes
>            Reporter: Gururajesh Elango
>            Priority: Major
>
> fs.s3a.path.style.access does not seem to work. Please see [https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro] to know about path style access
> How to reproduce:
> 1. Create a bucket in minio(S3 simulator)
> 2. Define a storage like below
> "storage": {
>  s3: {
>  type: "file",
>  connection: "s3a://new-bucket",
>  "config": {
>  "fs.s3a.access.key": "minio-user",
>  "fs.s3a.secret.key": "minio-password",
>  "fs.s3a.endpoint": "http://<minio-url>:9000",
>  "fs.s3a.connection.ssl.enabled": "false",
>  "fs.s3a.path.style.access": "true",
>  "fs.s3a.connection.timeout": "5000",
>  "fs.s3a.connection.maximum": "100"
>  },
>  "workspaces": {
>  "tmp": {
>  "location": "/tmp/drill",
>  "writable": "true",
>  "defaultInputFormat": "",
>  "allowAccessOutsideWorkspace": "false"
>  },
>  "root": {
>  "location": "/",
>  "writable": "false",
>  "defaultInputFormat": "",
>  "allowAccessOutsideWorkspace": "false"
>  }
>  },
>  "formats": {
>  "parquet": {
>  "type": "parquet"
>  },
>  "json": {
>  "type": "json",
>  "extensions": [
>  "json"
>  ]
>  },
>  "avro": {
>  "type": "avro"
>  }
>  },
>  "enabled": "true"
>  }
> }
> 3. In the logs, Expect an error which states
> new-bucket.<minio-url>:9000 is not reachable but what is expected is that Drill tries to reach
> http://<minio-url>:9000/impact-enable-bucket



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)