You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2016/02/22 17:06:18 UTC
[jira] [Commented] (HADOOP-12831) FSOutputSummer NPEs in ctor if
bytes per checksum set to 0
[ https://issues.apache.org/jira/browse/HADOOP-12831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15157199#comment-15157199 ]
Steve Loughran commented on HADOOP-12831:
-----------------------------------------
I had hoped this would be a way to disable checksumming and buffering on file:, but it instead I found a new way to break things
{code}
contains 1 event(s)
java.lang.NullPointerException
at org.apache.hadoop.fs.FSOutputSummer.<init>(FSOutputSummer.java:54)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:390)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:435)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:917)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:898)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:795)
at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFD.createLogFileStream(FileSystemTimelineWriter.java:406)
at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFD.prepareForWrite(FileSystemTimelineWriter.java:386)
at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFD.<init>(FileSystemTimelineWriter.java:363)
at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$EntityLogFD.<init>(FileSystemTimelineWriter.java:329)
at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFDsCache.createSummaryFDAndWrite(FileSystemTimelineWriter.java:842)
at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFDsCache.writeSummmaryEntityLogs(FileSystemTimelineWriter.java:826)
at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFDsCache.writeSummaryEntityLogs(FileSystemTimelineWriter.java:805)
at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter.putEntities(FileSystemTimelineWriter.java:222)
at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:658)
at org.apache.spark.deploy.history.yarn.YarnHistoryService.postOneEntity(YarnHistoryService.scala:825)
at org.apache.spark.deploy.history.yarn.YarnHistoryService.org$apache$spark$deploy$history$yarn$YarnHistoryService$$postEntities(YarnHistoryService.scala:899)
at org.apache.spark.deploy.history.yarn.YarnHistoryService$EntityPoster.run(YarnHistoryService.scala:1105)
{code}
> FSOutputSummer NPEs in ctor if bytes per checksum set to 0
> -----------------------------------------------------------
>
> Key: HADOOP-12831
> URL: https://issues.apache.org/jira/browse/HADOOP-12831
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Priority: Minor
>
> If you set the number of bytes per checksum to zero,
> {code} conf.setInt(LocalFileSystemConfigKeys.LOCAL_FS_BYTES_PER_CHECKSUM_KEY, 0)
> {code}
> then create a "file://" instance, you get to see a stack trace
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)