[GitHub] [druid] a2l007 commented on a change in pull request #10264: Fix CombiningFirehose compatibility

[GitBox <gi...@apache.org>]

a2l007 commented on a change in pull request #10264:
URL: https://github.com/apache/druid/pull/10264#discussion_r472243942



##########
File path: server/src/main/java/org/apache/druid/segment/realtime/firehose/CombiningFirehoseFactory.java
##########
@@ -33,11 +36,12 @@
 import java.io.IOException;
 import java.util.Iterator;
 import java.util.List;
+import java.util.stream.Stream;
 
 /**
  * Creates firehose that combines data from different Firehoses. Useful for ingesting data from multiple sources.
  */
-public class CombiningFirehoseFactory implements FirehoseFactory<InputRowParser>
+public class CombiningFirehoseFactory implements FiniteFirehoseFactory<InputRowParser, List<FirehoseFactory>>

Review comment:
       I feel that adding a non-splittable adapter would apply only to `CombiningFirehoseFactory`.  `ClippedFirehoseFactory` , `TimedShutoffFirehoseFactory` and `FixedCountFirehoseFactory` can be potentially splittable if the delegate `FirehoseFactory` is splittable, it's just that they haven't been fixed yet. 
   What we actually need is for index_parallel to support nested InputSource say DelegateInputSource. The delegate FirehoseFactories such as the ones listed above can then be easily provided their equivalent InputSource implementations. I've been looking at adding this support and hopefully can have a proposal out once I have something.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org