[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

["Colin Patrick McCabe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14000612#comment-14000612 ] 

Colin Patrick McCabe commented on SPARK-1518:
---------------------------------------------

{{FSDataOutputStream::sync()}} has been deprecated for a while, and finally got removed in HADOOP-8124.  It was a synonym for {{hflush}}, so replacing it with that function would probably be more appropriate.  You usually don't want {{hsync}}, since it forces an {{fsync}} on each datanode that contains the file.

It looks like there's some other YARN-related stuff that still fails to build once I make this fix, though...

> Spark master doesn't compile against hadoop-common trunk
> --------------------------------------------------------
>
>                 Key: SPARK-1518
>                 URL: https://issues.apache.org/jira/browse/SPARK-1518
>             Project: Spark
>          Issue Type: Bug
>            Reporter: Marcelo Vanzin
>
> FSDataOutputStream::sync() has disappeared from trunk in Hadoop; FileLogger.scala is calling it.
> I've changed it locally to hsync() so I can compile the code, but haven't checked yet whether those are equivalent. hsync() seems to have been there forever, so it hopefully works with all versions Spark cares about.



--
This message was sent by Atlassian JIRA
(v6.2#6252)