You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by "Keith Turner (Updated) (JIRA)" <ji...@apache.org> on 2012/02/17 21:50:57 UTC
[jira] [Updated] (ACCUMULO-417) unexpected exception in Split/MajC
following importDirectory call
[ https://issues.apache.org/jira/browse/ACCUMULO-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keith Turner updated ACCUMULO-417:
----------------------------------
Affects Version/s: (was: 1.5.0)
1.5.0-SNAPSHOT
Fix Version/s: 1.5.0
Great bug find. The bug does not exist in 1.4.x its a result of work done for ACCUMULO-288
> unexpected exception in Split/MajC following importDirectory call
> -----------------------------------------------------------------
>
> Key: ACCUMULO-417
> URL: https://issues.apache.org/jira/browse/ACCUMULO-417
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
> Affects Versions: 1.5.0-SNAPSHOT
> Environment: running 1.5.0-SNAPSHOT from svn on cluster of 9 8-core linux boxes running Centos 5.6. Hadoop 1.0 with zookeeper 3.3.3
> Reporter: Edward Seidl
> Assignee: Keith Turner
> Fix For: 1.5.0
>
>
> While attempting a bulk ingest from a mapreduce job, I noticed that after calling importDirectory() I started getting errors in the tservers like the following:
> {quote}
> 16 11:04:53,337 [file.FileUtil] DEBUG: Too many indexes (31) to open at once for [snip...], reducing in tmpDir = /accumulo/tmp/idxReduce_2009963461
> 16 11:04:53,595 [tabletserver.TabletServer] ERROR: Unexpected exception in Split/MajC initiator
> java.lang.NullPointerException
> at org.apache.accumulo.core.file.rfile.RFile$Writer.append(RFile.java:382)
> at org.apache.accumulo.core.file.FileUtil.reduceFiles(FileUtil.java:147)
> at org.apache.accumulo.core.file.FileUtil.findMidPoint(FileUtil.java:281)
> at org.apache.accumulo.core.file.FileUtil.findMidPoint(FileUtil.java:186)
> at org.apache.accumulo.server.tabletserver.Tablet.findSplitRow(Tablet.java:2939)
> at org.apache.accumulo.server.tabletserver.Tablet.needsSplit(Tablet.java:3013)
> at org.apache.accumulo.server.tabletserver.TabletServer$MajorCompactor.run(TabletServer.java:2066)
> at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
> at java.lang.Thread.run(Thread.java:619)
> {quote}
> As a result, my data was never showing up in the tables. I poked around in RFile.java, and noticed
> that the null pointer was the currentLocalityGroup. To get past this, I threw in a call to
> startDefaultLocalityGroup() if currentLocalityGroup is null (in RFile.append()).
> This then lead to the following error
> {quote}
> 6 15:15:46,989 [file.FileUtil] DEBUG: Too many indexes (40) to open at once for 10.252.
> 158.124 10.251.213.245:537, reducing in tmpDir = /accumulo/tmp/idxReduce_193905614116 15:15:48,060 [file.FileUtil] DEBUG: Finished reducing indexes for 10.252.158.124 10.2
> 51.213.245:537 in 1.07 secs16 15:15:48,068 [tabletserver.TabletServer] ERROR: Unexpected exception in Split/MajC initiator
> java.lang.IllegalArgumentException: File name rf_0000 has no extension
> at org.apache.accumulo.core.file.DispatchingFileFactory.findFileFactory(FileOperations.java:51)
> at org.apache.accumulo.core.file.DispatchingFileFactory.openIndex(FileOperations.java:67)
> at org.apache.accumulo.core.file.FileUtil.countIndexEntries(FileUtil.java:392)
> at org.apache.accumulo.core.file.FileUtil.findMidPoint(FileUtil.java:294)
> at org.apache.accumulo.core.file.FileUtil.findMidPoint(FileUtil.java:186)
> at org.apache.accumulo.server.tabletserver.Tablet.findSplitRow(Tablet.java:2939)
> at org.apache.accumulo.server.tabletserver.Tablet.needsSplit(Tablet.java:3013)
> at org.apache.accumulo.server.tabletserver.TabletServer$MajorCompactor.run(TabletServer.java:2066)
> at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
> at java.lang.Thread.run(Thread.java:619)
> {quote}
> To get past this one, I threw a ".rf" extension on the file being opened
> (outFile in FileUtil.reduceFiles()), and I also changed the add call
> immediately after from outFiles.add(newMapFile) to outFiles.add(outFile).
> Now my bulk imports work again. Don't know why this happens, and this
> surely isn't the proper way to fix the problem, but thought I'd let you
> know.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira