[GitHub] [incubator-druid] jihoonson commented on issue #8835: Ingestion task fails with NullPointerException during BUILD_SEGMENTS phase

[GitBox <gi...@apache.org>]

jihoonson commented on issue #8835: Ingestion task fails with NullPointerException during BUILD_SEGMENTS phase
URL: https://github.com/apache/incubator-druid/issues/8835#issuecomment-566839321
 
 
   @mlubavin-vg @anshbansal thank you for the report! I'm looking at the bug but unfortunately, it's not obvious what could be null here. The NPE happens at the below line:
   
   ```java
               try (InputStream inputStream = objectOpenFunction.open(object);
   ```
   
   `objectOpenFunction` cannot be null here since it's always created [here](https://github.com/apache/incubator-druid/blob/master/core/src/main/java/org/apache/druid/data/input/impl/prefetch/PrefetchableTextFilesFirehoseFactory.java#L206). If `object` was null, NPE should happen at a different line inside `open()`. I think I need more time to look at this. In the meantime, you can try using the [new inputSource and inputFormat](https://github.com/apache/incubator-druid/blob/master/docs/development/extensions-core/s3.md#s3-batch-ingestion-input-source). They are more intuitive than the existing firehose and more flexible to extend. If you still want to use the firehose, you can turn off prefetch by setting ` maxFetchCapacityBytes` to 0 (prefetch isn't that useful and not supported for the new inputSource implementation).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org