[jira] [Resolved] (HADOOP-14142) S3A - Adding unexpected prefix

["Steve Loughran (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-14142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran resolved HADOOP-14142.
-------------------------------------
    Resolution: Invalid

And closing as invalid. It's your home directory, which is the default unless you put a trailing "/" on the path. 

As stated before, If you are trying to show your S3 implementation works with hadoop, run the hadoop-aws test suite first. For spark, [try these tests|https://github.com/steveloughran/spark-cloud-examples]

Either way, immediately panicing with critical bug reports isn't going to fix the problem. You've got partway there with the logging, but take advantage of the code: step through it to see what's happening. It's what we end up doing

> S3A - Adding unexpected prefix
> ------------------------------
>
>                 Key: HADOOP-14142
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14142
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 2.7.3
>            Reporter: Vishnu Vardhan
>            Priority: Minor
>
> Hi:
> S3A seems to prefix unexpected prefix to my s3 path
> Specifically, in the debug log below the following line is unexpected
> >  GET /myBkt8/?max-keys=1&prefix=user%2Fvardhan%2F&delimiter=%2F HTTP/1.1
> It is not clear where the "prefix" is coming from and why.
> I executed the following commands
> sc.setLogLevel("DEBUG")
> sc.hadoopConfiguration.set("fs.s3a.impl","org.apache.hadoop.fs.s3a.S3AFileSystem")
> sc.hadoopConfiguration.set("fs.s3a.endpoint","webscaledemo.netapp.com:8082")
> sc.hadoopConfiguration.set("fs.s3a.access.key","")
> sc.hadoopConfiguration.set("fs.s3a.secret.key","")
> sc.hadoopConfiguration.set("fs.s3a.path.style.access","false")
> val s3Rdd = sc.textFile("s3a://myBkt98")
> s3Rdd.count()
> ----



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org