[jira] [Commented] (IOTDB-3961) bug for Tsfile compact

["Bo Zhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IOTDB-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571168#comment-17571168 ] 

Bo Zhang commented on IOTDB-3961:
---------------------------------

root cause is : when report Metric to Prometheus, it triggers exception, which casue CompactionWorker.run exists and can not recover.

=================================================================

2022-07-25 17:49:12,907 [pool-8-IoTDB-Compaction-3] ERROR o.a.i.d.e.c.CompactionWorker:80 - run(), CompactionWorker exception.

java.lang.IllegalArgumentException: Prometheus requires that all meters with the same name have the same set of tag keys. There is already an existing meter named 'compaction_task_count' containing tag keys [name]. The meter you are attempting to register has keys [name, type].

        at io.micrometer.prometheus.PrometheusMeterRegistry.lambda$throwExceptionOnRegistrationFailure$17(PrometheusMeterRegistry.java:444)

        at io.micrometer.core.instrument.MeterRegistry.meterRegistrationFailed(MeterRegistry.java:1028)

        at io.micrometer.prometheus.PrometheusMeterRegistry.lambda$applyToCollector$16(PrometheusMeterRegistry.java:419)

        at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1877)

        at io.micrometer.prometheus.PrometheusMeterRegistry.applyToCollector(PrometheusMeterRegistry.java:406)

        at io.micrometer.prometheus.PrometheusMeterRegistry.newCounter(PrometheusMeterRegistry.java:109)

        at io.micrometer.core.instrument.MeterRegistry.lambda$registerMeterIfNecessary$5(MeterRegistry.java:561)

        at io.micrometer.core.instrument.MeterRegistry.getOrCreateMeter(MeterRegistry.java:614)

        at io.micrometer.core.instrument.MeterRegistry.registerMeterIfNecessary(MeterRegistry.java:568)

        at io.micrometer.core.instrument.MeterRegistry.registerMeterIfNecessary(MeterRegistry.java:561)

        at io.micrometer.core.instrument.MeterRegistry.counter(MeterRegistry.java:284)

        at io.micrometer.core.instrument.Counter$Builder.register(Counter.java:128)

        at io.micrometer.core.instrument.composite.CompositeCounter.registerNewMeter(CompositeCounter.java:50)

        at io.micrometer.core.instrument.composite.CompositeCounter.registerNewMeter(CompositeCounter.java:23)

        at io.micrometer.core.instrument.composite.AbstractCompositeMeter.add(AbstractCompositeMeter.java:66)

        at java.lang.Iterable.forEach(Iterable.java:75)

        at java.util.Collections$SetFromMap.forEach(Collections.java:5530)

        at io.micrometer.core.instrument.composite.CompositeMeterRegistry.lambda$new$0(CompositeMeterRegistry.java:65)

        at io.micrometer.core.instrument.composite.CompositeMeterRegistry.lock(CompositeMeterRegistry.java:184)

        at io.micrometer.core.instrument.composite.CompositeMeterRegistry.lambda$new$1(CompositeMeterRegistry.java:65)

        at io.micrometer.core.instrument.MeterRegistry.getOrCreateMeter(MeterRegistry.java:624)

        at io.micrometer.core.instrument.MeterRegistry.registerMeterIfNecessary(MeterRegistry.java:568)

        at io.micrometer.core.instrument.MeterRegistry.registerMeterIfNecessary(MeterRegistry.java:561)

        at io.micrometer.core.instrument.MeterRegistry.counter(MeterRegistry.java:284)

        at io.micrometer.core.instrument.Counter$Builder.register(Counter.java:128)

        at io.micrometer.core.instrument.MeterRegistry.counter(MeterRegistry.java:391)

        at io.micrometer.core.instrument.MeterRegistry.counter(MeterRegistry.java:402)

        at org.apache.iotdb.metrics.micrometer.MicrometerMetricManager.lambda$count$7(MicrometerMetricManager.java:218)

        at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)

        at org.apache.iotdb.metrics.micrometer.MicrometerMetricManager.count(MicrometerMetricManager.java:217)

        at org.apache.iotdb.db.service.metrics.recorder.CompactionMetricsRecorder.recordTaskInfo(CompactionMetricsRecorder.java:151)

        at org.apache.iotdb.db.engine.compaction.CompactionTaskManager.removeRunningTaskFuture(CompactionTaskManager.java:286)

        at org.apache.iotdb.db.engine.compaction.task.AbstractCompactionTask.start(AbstractCompactionTask.java:81)

        at org.apache.iotdb.db.engine.compaction.CompactionWorker.run(CompactionWorker.java:75)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

        at java.lang.Thread.run(Thread.java:748)

> bug for Tsfile compact
> ----------------------
>
>                 Key: IOTDB-3961
>                 URL: https://issues.apache.org/jira/browse/IOTDB-3961
>             Project: Apache IoTDB
>          Issue Type: Bug
>          Components: Core/Compaction
>            Reporter: Bo Zhang
>            Priority: Major
>
> find 1 critial bug during  long term stabliity testing.
> After 4 hours stabliity testing, the tsfiles' number will become bigger and bigger.
>  
> !https://aone.alibaba-inc.com/v2/api/workitem/adapter/file/url?fileIdentifier=workitem%2Falibaba%2Fdefault%2F1658719978006image.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)