[jira] [Commented] (SPARK-8439) SPARK-7155 - supporting comma separated paths, breaks compatibility

["Eugen Cepoi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-8439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591675#comment-14591675 ] 

Eugen Cepoi commented on SPARK-8439:
------------------------------------

Lets continue the discussion here.
I agree with having a single way to do things is cleaner. My point is mostly about regression that breaks code in some tricky way.
I could detect it due to some unit tests I had, but this might break others code in production.

I have a fix ready that replaces all the setInputPaths by addInputPaths. But if you feel that it's better to prevent people from adding paths from other places then I am fine with closing this issue.

> SPARK-7155 - supporting comma separated paths, breaks compatibility
> -------------------------------------------------------------------
>
>                 Key: SPARK-8439
>                 URL: https://issues.apache.org/jira/browse/SPARK-8439
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.4.0
>            Reporter: Eugen Cepoi
>
> The improvement to support comma separated paths, replaced code that was adding paths to the config by code that is setting them.
> This breaks user code that added by hand the paths using FileInputFormat.addInputPath(...).
> The fix would be to use addInputPaths instead, so people can still add the paths using other mechanisms instead of constructing comma separated strings.
> The PR that introduces the regression: https://github.com/apache/spark/pull/5708



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org