You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/02/21 11:47:35 UTC
[GitHub] [druid] gunjdesai opened a new issue #9389:
CombiningFirehoseFactory cannot be cast to FiniteFirehoseFactory
gunjdesai opened a new issue #9389: CombiningFirehoseFactory cannot be cast to FiniteFirehoseFactory
URL: https://github.com/apache/druid/issues/9389
I am attempting to combing two datasources using the CombiningFirehoseFactory
For reference, I am following the [update data tutorial](https://druid.apache.org/docs/latest/tutorials/tutorial-update-data.html)
My ingestion spec looks like this
`{
"type" : "index",
"spec" : {
"dataSchema" : {
"dataSource" : "updates-tutorial",
"parser" : {
"type" : "string",
"parseSpec" : {
"format" : "json",
"dimensionsSpec" : {
"dimensions" : [
"animal"
]
},
"timestampSpec": {
"column": "timestamp",
"format": "iso"
}
}
},
"metricsSpec" : [
{ "type" : "count", "name" : "count" },
{ "type" : "longSum", "name" : "number", "fieldName" : "number" }
],
"granularitySpec" : {
"type" : "uniform",
"segmentGranularity" : "hour",
"queryGranularity" : "minute",
"intervals" : ["2018-01-01/2018-01-03"],
"rollup" : false
}
},
"ioConfig" : {
"type" : "index",
"firehose" : {
"type": "combining",
"delegates": [
{
"type" : "ingestSegment",
"dataSource" : "s1",
"interval" : "2018-01-01/2018-01-03"
},
{
"type" : "ingestSegment",
"dataSource" : "s2",
"interval" : "2018-01-01/2018-01-03"
}
]
},
"appendToExisting" : false
},
"tuningConfig" : {
"type" : "index",
"maxRowsPerSegment" : 5000000,
"maxRowsInMemory" : 25000
}
}
}`
After reading into the code a bit, I believe that CombiningFirehoseFactory allows you to combine multiple datasources
But when i submit the task, i get the following error
`2020-02-21T11:35:39,222 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.common.task.IndexTask - Encountered exception in NOT_STARTED.
java.lang.ClassCastException: org.apache.druid.segment.realtime.firehose.CombiningFirehoseFactory cannot be cast to org.apache.druid.data.input.FiniteFirehoseFactory
at org.apache.druid.indexing.common.task.IndexTask$IndexIOConfig.getNonNullInputSource(IndexTask.java:1148) ~[druid-indexing-service-0.17.0.jar:0.17.0]
at org.apache.druid.indexing.common.task.IndexTask.runTask(IndexTask.java:477) [druid-indexing-service-0.17.0.jar:0.17.0]
at org.apache.druid.indexing.common.task.AbstractBatchIndexTask.run(AbstractBatchIndexTask.java:138) [druid-indexing-service-0.17.0.jar:0.17.0]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:419) [druid-indexing-service-0.17.0.jar:0.17.0]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:391) [druid-indexing-service-0.17.0.jar:0.17.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]`
I am not sure, if this is a issue with the datasource of my selection or config issue.
In the interim, i have tried to alternate the datasource from `ingestSegment` to `local` as well, but i continue to get the same error.
Would really appreciate if i could get any help on this or if someone can help rectify my understanding on the matter.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] a2l007 commented on issue #9389: CombiningFirehoseFactory
cannot be cast to FiniteFirehoseFactory
Posted by GitBox <gi...@apache.org>.
a2l007 commented on issue #9389: CombiningFirehoseFactory cannot be cast to FiniteFirehoseFactory
URL: https://github.com/apache/druid/issues/9389#issuecomment-589692443
It looks like a bug since `CombiningFirehoseFactory` should be implementing `FiniteFirehoseFactory` and there isn't an equivalent `InputSource` implementation for this firehose yet.
Can you try directly using the `ingestSegment` firehose instead of going through the combining firehose?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] 9553973 commented on issue #9389: CombiningFirehoseFactory
cannot be cast to FiniteFirehoseFactory
Posted by GitBox <gi...@apache.org>.
9553973 commented on issue #9389: CombiningFirehoseFactory cannot be cast to FiniteFirehoseFactory
URL: https://github.com/apache/druid/issues/9389#issuecomment-593069172
FYI - I encountered the exact same issue following the tutorial, when submit task which
bin/post-index-task --file quickstart/tutorial/updates-append-index.json --url http://localhost:8081.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gunjdesai commented on issue #9389:
CombiningFirehoseFactory cannot be cast to FiniteFirehoseFactory
Posted by GitBox <gi...@apache.org>.
gunjdesai commented on issue #9389: CombiningFirehoseFactory cannot be cast to FiniteFirehoseFactory
URL: https://github.com/apache/druid/issues/9389#issuecomment-591845307
Hey,
Sorry i couldn't respond on this earlier.
I tried using the `ingestSegment` instead of using combining firehose.
It's rejected as a bad request with the following error
```
Exception: HTTP Error 400: Bad Request, check overlord log for more details.
{"error":"Cannot construct instance of `org.apache.druid.indexing.firehose.IngestSegmentFirehoseFactory`, problem: dataSource\n at [Source: (org.eclipse.jetty.server.HttpInputOverHTTP); line: 49, column: 7] (through reference chain: org.apache.druid.indexing.common.task.IndexTask[\"spec\"]->org.apache.druid.indexing.common.task.IndexTask$IndexIngestionSpec[\"ioConfig\"]->org.apache.druid.indexing.common.task.IndexTask$IndexIOConfig[\"firehose\"])"}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] a2l007 commented on issue #9389: CombiningFirehoseFactory
cannot be cast to FiniteFirehoseFactory
Posted by GitBox <gi...@apache.org>.
a2l007 commented on issue #9389: CombiningFirehoseFactory cannot be cast to FiniteFirehoseFactory
URL: https://github.com/apache/druid/issues/9389#issuecomment-592142693
@gunjdesai That exception is generally thrown when the indexing spec is incorrect. Could you double check your ingestion json with the format for `ingestSegment` firehose? Please post your ingestion spec here in case that didn't fix the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] aguereca commented on issue #9389: CombiningFirehoseFactory
cannot be cast to FiniteFirehoseFactory
Posted by GitBox <gi...@apache.org>.
aguereca commented on issue #9389: CombiningFirehoseFactory cannot be cast to FiniteFirehoseFactory
URL: https://github.com/apache/druid/issues/9389#issuecomment-592064006
FYI - I encountered the exact same issue following the tutorial, not what you want to see while evaluating a new technology, will keep an 👁 on this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jianxang commented on issue #9389: CombiningFirehoseFactory
cannot be cast to FiniteFirehoseFactory
Posted by GitBox <gi...@apache.org>.
jianxang commented on issue #9389: CombiningFirehoseFactory cannot be cast to FiniteFirehoseFactory
URL: https://github.com/apache/druid/issues/9389#issuecomment-596301501
I also facing the same issue when trying on their official tutorial when submitting the task: bin/post-index-task --file quickstart/tutorial/updates-append-index.json --url http://localhost:8081
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org