[jira] [Commented] (SPARK-7481) Add spark-cloud module to pull in object store support; test

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15582210#comment-15582210 ] 

Steve Loughran commented on SPARK-7481:
---------------------------------------

For anyone watching this; the code is pretty much ready to go in as far as packaging, testing, docs go. 

SBT builds everything, but it doesn't run the tests...it's not just that the logic in the tests to skip suites isn't picking up the property reference to a configuration file, scalatest doesn't seem to be finding any tests under cloud/target/ ... even though they are being compiled in. Assistance there would be very welcome.

> Add spark-cloud module to pull in object store support; test
> ------------------------------------------------------------
>
>                 Key: SPARK-7481
>                 URL: https://issues.apache.org/jira/browse/SPARK-7481
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build
>    Affects Versions: 1.3.1
>            Reporter: Steve Loughran
>
> To keep the s3n classpath right, to add s3a, swift & azure, the dependencies of spark in a 2.6+ profile need to add the relevant object store packages (hadoop-aws, hadoop-openstack, hadoop-azure)
> this adds more stuff to the client bundle, but will mean a single spark package can talk to all of the stores.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org