You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "Michael Miller (JIRA)" <ji...@apache.org> on 2016/11/15 21:08:58 UTC
[jira] [Commented] (ACCUMULO-4486) Compaction from Shell hangs on
Unknown property
[ https://issues.apache.org/jira/browse/ACCUMULO-4486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668329#comment-15668329 ]
Michael Miller commented on ACCUMULO-4486:
------------------------------------------
After spending some time looking at this issue, it seems to be a limitation of the API. When the shell calls compact, it will wait for flush to complete before continuing. The flush will fail because of an unknown property but there is no way for the shell to know that this has happened. The compaction thread will then sleep for a random period of time only to keep failing when it tries again.
> Compaction from Shell hangs on Unknown property
> -----------------------------------------------
>
> Key: ACCUMULO-4486
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4486
> Project: Accumulo
> Issue Type: Bug
> Components: shell, tserver
> Affects Versions: 1.8.0
> Reporter: Michael Miller
> Assignee: Michael Miller
> Priority: Minor
>
> I was following the sampling [example|https://accumulo.apache.org/1.8/examples/sample] when I mistyped one of the table properties. After realizing this I entered the correct property but didn't remove the bad one. I then ran the "compact" command but the shell hung indefinitely. There was an IllegalArgumentException printed in the log but I believe the bug is how the shell (or Compactor) mishandled the exception, never returning control to the user. Eventually I did get a warning on the shell "WARN : Thread "shell" stuck on IO.." but never regained control of the shell. Here are the commands run:
> {code}
> createtable sampex
> insert 9255 doc content 'abcde'
> insert 9255 doc url file://foo.txt
> insert 8934 doc content 'accumulo scales'
> insert 8934 doc url file://accumulo_notes.txt
> insert 2317 doc content 'milk, eggs, bread, parmigiano-reggiano'
> insert 2317 doc url file://groceries/9.txt
> insert 3900 doc content 'EC2 ate my homework'
> insert 3900 doc uril file://final_project.txt
> config -t sampex -s table.sampler=org.apache.accumulo.core.client.sample.RowSampler
> config -t sampex -s table.sampler.opt.haser=murmur3_32
> config -t sampex -s table.sampler.opt.hasher=murmur3_32
> config -t sampex -s table.sampler.opt.modulus=3
> compact -t sampex --sf-no-sample
> {code}
> Stacktrace of the exception printed in the tserver log:
> {code}
> 2016-10-05 13:13:25,299 [tablet.Compactor] ERROR: Unknown option : haser
> java.lang.IllegalArgumentException: Unknown option : haser
> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
> at org.apache.accumulo.core.client.sample.AbstractHashSampler.init(AbstractHashSampler.java:81)
> at org.apache.accumulo.core.sample.impl.SamplerFactory.newSampler(SamplerFactory.java:40)
> at org.apache.accumulo.core.file.rfile.RFileOperations.openWriter(RFileOperations.java:91)
> at org.apache.accumulo.core.file.DispatchingFileFactory.openWriter(DispatchingFileFactory.java:74)
> at org.apache.accumulo.core.file.FileOperations$OpenWriterOperation.build(FileOperations.java:331)
> at org.apache.accumulo.tserver.tablet.Compactor.call(Compactor.java:201)
> at org.apache.accumulo.tserver.tablet.MinorCompactor.call(MinorCompactor.java:111)
> at org.apache.accumulo.tserver.tablet.Tablet.minorCompact(Tablet.java:811)
> at org.apache.accumulo.tserver.tablet.MinorCompactionTask.run(MinorCompactionTask.java:84)
> at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
> at org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
> at java.lang.Thread.run(Thread.java:745)
> 2016-10-05 13:13:25,299 [tablet.MinorCompactor] WARN : MinC failed (Unknown option : haser) to create hdfs://localhost:10000/accumulo/tables/2/default_tablet/F0000000.rf_tmp retrying ...
> java.lang.IllegalArgumentException: Unknown option : haser
> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
> at org.apache.accumulo.core.client.sample.AbstractHashSampler.init(AbstractHashSampler.java:81)
> at org.apache.accumulo.core.sample.impl.SamplerFactory.newSampler(SamplerFactory.java:40)
> at org.apache.accumulo.core.file.rfile.RFileOperations.openWriter(RFileOperations.java:91)
> at org.apache.accumulo.core.file.DispatchingFileFactory.openWriter(DispatchingFileFactory.java:74)
> at org.apache.accumulo.core.file.FileOperations$OpenWriterOperation.build(FileOperations.java:331)
> at org.apache.accumulo.tserver.tablet.Compactor.call(Compactor.java:201)
> at org.apache.accumulo.tserver.tablet.MinorCompactor.call(MinorCompactor.java:111)
> at org.apache.accumulo.tserver.tablet.Tablet.minorCompact(Tablet.java:811)
> at org.apache.accumulo.tserver.tablet.MinorCompactionTask.run(MinorCompactionTask.java:84)
> at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
> at org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> The thread will then sleep:
> 2016-10-05 13:13:25,324 [tablet.MinorCompactor] DEBUG: MinC failed sleeping 10116 ms before retrying
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)