You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/05/08 04:46:59 UTC
[GitHub] [druid] successok opened a new issue #11221: The task actually failed but the status shows success
successok opened a new issue #11221:
URL: https://github.com/apache/druid/issues/11221
### Affected Version
the latest version 0.21.0
### Description
As a loyal user of druid, I upgraded the version immediately after it was upgraded to 0.21.0, When I created a datasource from Kafka with old version spec(0.20.2),the task appears to has been completed successfully in task console, and Its status was success, but I can't find it both in the datasource interface and HDFS Deep Storage . From the end of task logs I found the following error message :
`2021-05-08T00:58:10,643 WARN [pool-5-thread-2] org.apache.druid.emitter.kafka.KafkaEmitter - Exception while getting record from queue or producer send, Events would not be emitted anymore.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_181]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_181]
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) ~[?:1.8.0_181]
at org.apache.druid.emitter.kafka.MemoryBoundLinkedBlockingQueue.take(MemoryBoundLinkedBlockingQueue.java:59) ~[?:?]
at org.apache.druid.emitter.kafka.KafkaEmitter.sendToKafka(KafkaEmitter.java:136) ~[?:?]
at org.apache.druid.emitter.kafka.KafkaEmitter.sendAlertToKafka(KafkaEmitter.java:128) ~[?:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_181]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_181]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_181]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]`
I know the new version has some changes in ingestion format ,the problem has been solved after reload data from kafka with new spec, but it seems the task status is inaccurate in this version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok edited a comment on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok edited a comment on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835113902
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok edited a comment on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok edited a comment on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835126320
the old spec is normal in 0.20.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok edited a comment on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok edited a comment on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835113902
Here is the spec information:
`{
"type": "kafka",
"context": {
"priority": 90,
"forceTimeChunkLock": true
},
"spec": {
"ioConfig": {
"type": "kafka",
"consumerProperties": {
"bootstrap.servers": "...."
},
"topic": "....",
"inputFormat": {
"type": "json"
},
"useEarliestOffset": true,
"appendToExisting": true,
"taskCount": 1,
"skipOffsetGaps": true,
"taskDuration": "PT1H",
"completionTimeout": "PT1H"
},
"tuningConfig": {
"type": "kafka",
"logParseExceptions": false
},
"dataSchema": {
"dataSource": "bi_data1",
"granularitySpec": {
"type": "uniform",
"queryGranularity": "HOUR",
"segmentGranularity": "HOUR",
"rollup": false
},
"timestampSpec": {
"column": "log_time_BI",
"format": "iso"
},
"dimensionsSpec": {
"dimensions": [
{
"type": "long",
"name": "days_diff"
},
{
"type": "string",
"name": "action_type"
},
{
"type": "string",
"name": "adgroup_name"
},
{
"type": "string",
"name": "adjustADID"
},
{
"type": "string",
"name": "adjustIDFA"
},
{
"type": "string",
"name": "adjustLabel"
},
{
"type": "string",
"name": "adjustTrackerToken"
},
{
"type": "string",
"name": "agent"
},
{
"type": "string",
"name": "androidId"
},
{
"type": "string",
"name": "app"
}
]
}
}
}
}`
While reproducing the problem ,i find it seems that druid cannot parse timestamp correctly from old version spec
[error.log](https://github.com/apache/druid/files/6445283/error.log)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok edited a comment on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok edited a comment on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835113902
Here is the spec information:
`{
"type": "kafka",
"context": {
"priority": 90,
"forceTimeChunkLock": true
},
"spec": {
"ioConfig": {
"type": "kafka",
"consumerProperties": {
"bootstrap.servers": "...."
},
"topic": "....",
"inputFormat": {
"type": "json"
},
"useEarliestOffset": true,
"appendToExisting": true,
"taskCount": 1,
"skipOffsetGaps": true,
"taskDuration": "PT1H",
"completionTimeout": "PT1H"
},
"tuningConfig": {
"type": "kafka",
"logParseExceptions": false
},
"dataSchema": {
"dataSource": "bi_data1",
"granularitySpec": {
"type": "uniform",
"queryGranularity": "HOUR",
"segmentGranularity": "HOUR",
"rollup": false
},
"timestampSpec": {
"column": "log_time_BI",
"format": "iso"
},
"dimensionsSpec": {
"dimensions": [
{
"type": "long",
"name": "days_diff"
},
{
"type": "string",
"name": "action_type"
},
{
"type": "string",
"name": "adgroup_name"
},
{
"type": "string",
"name": "adjustADID"
},
{
"type": "string",
"name": "adjustIDFA"
},
{
"type": "string",
"name": "adjustLabel"
},
{
"type": "string",
"name": "adjustTrackerToken"
},
{
"type": "string",
"name": "agent"
},
{
"type": "string",
"name": "androidId"
},
{
"type": "string",
"name": "app"
}
]
}
}
}
}`
While reproducing the problem ,i find it seems that druid cannot parse '__time' correctly from old version spec
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok closed issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok closed issue #11221:
URL: https://github.com/apache/druid/issues/11221
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok commented on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok commented on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835412749
The problem was resolved when the messages of Zookeeper was deleted and the cluster was restarted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok edited a comment on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok edited a comment on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835113902
Here is the spec information:
`{
"type": "kafka",
"context": {
"priority": 90,
"forceTimeChunkLock": true
},
"spec": {
"ioConfig": {
"type": "kafka",
"consumerProperties": {
"bootstrap.servers": "...."
},
"topic": "....",
"inputFormat": {
"type": "json"
},
"useEarliestOffset": true,
"appendToExisting": true,
"taskCount": 1,
"skipOffsetGaps": true,
"taskDuration": "PT1H",
"completionTimeout": "PT1H"
},
"tuningConfig": {
"type": "kafka",
"logParseExceptions": false
},
"dataSchema": {
"dataSource": "bi_data1",
"granularitySpec": {
"type": "uniform",
"queryGranularity": "HOUR",
"segmentGranularity": "HOUR",
"rollup": false
},
"timestampSpec": {
"column": "log_time_BI",
"format": "iso"
},
"dimensionsSpec": {
"dimensions": [
{
"type": "long",
"name": "days_diff"
},
{
"type": "string",
"name": "action_type"
},
{
"type": "string",
"name": "adgroup_name"
},
{
"type": "string",
"name": "adjustADID"
},
{
"type": "string",
"name": "adjustIDFA"
},
{
"type": "string",
"name": "adjustLabel"
},
{
"type": "string",
"name": "adjustTrackerToken"
},
{
"type": "string",
"name": "agent"
},
{
"type": "string",
"name": "androidId"
},
{
"type": "string",
"name": "app"
}
]
}
}
}
}`
While reproducing the problem ,i find it seems that druid cannot parse timestamp correctly from old version spec
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok commented on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok commented on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835126320
the old spec is normal in 0.20.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok commented on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok commented on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835113902
here is the spec information:
`{
"type": "kafka",
"context": {
"priority": 90,
"forceTimeChunkLock": true
},
"spec": {
"ioConfig": {
"type": "kafka",
"consumerProperties": {
"bootstrap.servers": "...."
},
"topic": "....",
"inputFormat": {
"type": "json"
},
"useEarliestOffset": true,
"appendToExisting": true,
"taskCount": 1,
"skipOffsetGaps": true,
"taskDuration": "PT1H",
"completionTimeout": "PT1H"
},
"tuningConfig": {
"type": "kafka",
"logParseExceptions": false
},
"dataSchema": {
"dataSource": "bi_data1",
"granularitySpec": {
"type": "uniform",
"queryGranularity": "HOUR",
"segmentGranularity": "HOUR",
"rollup": false
},
"timestampSpec": {
"column": "log_time_BI",
"format": "iso"
},
"dimensionsSpec": {
"dimensions": [
{
"type": "long",
"name": "days_diff"
},
{
"type": "string",
"name": "action_type"
},
{
"type": "string",
"name": "adgroup_name"
},
{
"type": "string",
"name": "adjustADID"
},
{
"type": "string",
"name": "adjustIDFA"
},
{
"type": "string",
"name": "adjustLabel"
},
{
"type": "string",
"name": "adjustTrackerToken"
},
{
"type": "string",
"name": "agent"
},
{
"type": "string",
"name": "androidId"
},
{
"type": "string",
"name": "app"
}
]
}
}
}
}`
while reproducing the problem ,i find it seems druid cannot parse timestamp correctly from old version spec
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok edited a comment on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok edited a comment on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835126320
The old spec is normal in druid 0.20.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok edited a comment on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok edited a comment on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835113902
Here is the spec information:
`{
"type": "kafka",
"context": {
"priority": 90,
"forceTimeChunkLock": true
},
"spec": {
"ioConfig": {
"type": "kafka",
"consumerProperties": {
"bootstrap.servers": "...."
},
"topic": "....",
"inputFormat": {
"type": "json"
},
"useEarliestOffset": true,
"appendToExisting": true,
"taskCount": 1,
"skipOffsetGaps": true,
"taskDuration": "PT1H",
"completionTimeout": "PT1H"
},
"tuningConfig": {
"type": "kafka",
"logParseExceptions": false
},
"dataSchema": {
"dataSource": "bi_data1",
"granularitySpec": {
"type": "uniform",
"queryGranularity": "HOUR",
"segmentGranularity": "HOUR",
"rollup": false
},
"timestampSpec": {
"column": "log_time_BI",
"format": "iso"
},
"dimensionsSpec": {
"dimensions": [
{
"type": "long",
"name": "days_diff"
},
{
"type": "string",
"name": "action_type"
},
{
"type": "string",
"name": "adgroup_name"
},
{
"type": "string",
"name": "adjustADID"
},
{
"type": "string",
"name": "adjustIDFA"
},
{
"type": "string",
"name": "adjustLabel"
},
{
"type": "string",
"name": "adjustTrackerToken"
},
{
"type": "string",
"name": "agent"
},
{
"type": "string",
"name": "androidId"
},
{
"type": "string",
"name": "app"
}
]
}
}
}
}`
While reproducing the problem ,i find it seems that druid cannot parse '__time' correctly from old version spec
[error.log](https://github.com/apache/druid/files/6445283/error.log)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] successok commented on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
successok commented on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835123579
This is the new spec after reloading data from kafka which is normal:
`{"type":"kafka","spec":{"ioConfig":{"type":"kafka","consumerProperties":{"bootstrap.servers":"....."},"topic":".....","inputFormat":{"type":"json"},"useEarliestOffset":true},"tuningConfig":{"type":"kafka","maxRowsPerSegment":30000000,"maxTotalRows":40000000},"dataSchema":{"dataSource":".....","timestampSpec":{"column":"log_time_BI","format":"auto"},"dimensionsSpec":{"dimensions":[{"type":"long","name":"days_diff"},{"type":"string","name":"action_type"},{"type":"string","name":"adgroup_name"},{"type":"string","name":"adjustADID"}]},"granularitySpec":{"queryGranularity":"none","rollup":false,"segmentGranularity":"hour"},"transformSpec":{"transforms":[{"type":"expression","name":"install_timestamp","expression":"timestamp(install_time)"}]}}}}`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on issue #11221: The task actually failed but the status shows success
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on issue #11221:
URL: https://github.com/apache/druid/issues/11221#issuecomment-835103686
The exception is thrown from KafkaEmitter, not the ingestion, so the ingest task does not fail.
You could upload the entire task log so that we could know why there's such an exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org