You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2011/03/18 19:34:29 UTC
[jira] Commented: (CASSANDRA-2346) Windows: SSTableWriterTest Fails
[ https://issues.apache.org/jira/browse/CASSANDRA-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008566#comment-13008566 ]
Jonathan Ellis commented on CASSANDRA-2346:
-------------------------------------------
My guess is, this part of closeAndOpenReader just prior to the delete attempt is responsible:
{code}
SegmentedFile ifile = iwriter.builder.complete(newdesc.filenameFor(SSTable.COMPONENT_INDEX));
SegmentedFile dfile = dbuilder.complete(newdesc.filenameFor(SSTable.COMPONENT_DATA));
SSTableReader sstable = SSTableReader.internalOpen(newdesc, components, metadata, partitioner, ifile, dfile, iwriter.summary, iwriter.bf, maxDataAge, estimatedRowSize, estimatedColumnCount);
{code}
In particular, iwriter.builder.complete mmap's the index file, which for Windows also counts as having an open fd. A reference to the mmap'd segments is retained by the returned sstable.
A nasty hack that MIGHT work (depending on GC to clean things up is awful) would be to change the test to something like
{code}
// avoid retaining a reference to the returned sstable
Descriptor descriptor = SSTableUtils.prepare().cf("Indexed1").writeRaw(entries).descriptor;
// clean up mmap'd segments in the reader we just created so Windows allows deleting the backing file
System.gc();
// whack the index to trigger the recover
FileUtils.deleteWithConfirm(descriptor.filenameFor(Component.PRIMARY_INDEX));
FileUtils.deleteWithConfirm(descriptor.filenameFor(Component.FILTER));
{code}
A cleaner approach would be to ditch SSTableUtils (not my favorite class to begin with), expose a method in SSTWriter that just does data appends (no index) and create just the data file that way, instead of trying to delete the supporting files later.
An in-between approach might be to create a SSTW close() method that just returns the descriptor after shutting things down, without opening the problematic reader. Then the test would work as-is.
> Windows: SSTableWriterTest Fails
> --------------------------------
>
> Key: CASSANDRA-2346
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2346
> Project: Cassandra
> Issue Type: Bug
> Components: Tests
> Affects Versions: 0.7.4
> Environment: Windows
> Reporter: Benjamin Coverston
> Labels: windows
> Fix For: 0.7.5
>
>
> Fails with the following exception[1]
> Which looks like there's a stream keeping this file open somewhere.
> I looked for the culprit, but I was unable to identify it. Following it through it appears that if the following two lines of code are executed from IndexWriter.afterAppend:
> ByteBufferUtil.writeWithShortLength(key.key, indexFile);
> indexFile.writeLong(dataPosition);
> Then the indexFile gets touched and that generally triggers this situation. One thing interesting here is that an index file without any rows will trigger a divide by zero exception [2].
> I looked at this quite a bit without being able to make much progress.
> [1]
> java.io.IOException: Failed to delete C:\Users\Ben\AppData\Local\Temp\Keyspace12505092819180437091Indexed1\Keyspace1\Indexed1-f-0-Index.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTableWriterTest.testRecoverAndOpen(SSTableWriterTest.java:79)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:97)
> at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:196)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:115)
> [2]
> ERROR 16:57:48,119 Error in ThreadPoolExecutor
> java.lang.ArithmeticException: / by zero
> at org.apache.cassandra.io.sstable.SSTable.estimateRowsFromData(SSTable.java:218)
> at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:291)
> at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:942)
> at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:935)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> java.util.concurrent.ExecutionException: java.lang.ArithmeticException: / by zero
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.io.sstable.SSTableWriterTest.testRecoverAndOpen(SSTableWriterTest.java:82)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:97)
> at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:196)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:115)
> Caused by: java.lang.ArithmeticException: / by zero
> at org.apache.cassandra.io.sstable.SSTable.estimateRowsFromData(SSTable.java:218)
> at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:291)
> at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:942)
> at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:935)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira