You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2019/02/02 22:55:40 UTC
[GitHub] anantmf commented on issue #6980: With latest version of druid is
it possible to use OpenStack Swift custom implementation (not Amazon S3)?
anantmf commented on issue #6980: With latest version of druid is it possible to use OpenStack Swift custom implementation (not Amazon S3)?
URL: https://github.com/apache/incubator-druid/issues/6980#issuecomment-460006580
Hi @jihoonson , Thanks for the reply. I found couple of configuration issues after which I was able to make it work. So now segments are correctly stored in Swift, however when Historical is trying to get it back from Deep storage it fails.
I think the issue is
https://github.com/apache/incubator-druid/pull/5382#discussion_r175933001
When it is doing ListObjectsV2Request, i tried to debug the code and the response actually has object summary but KeyCount is Zero. Which is causing it to fail.
I tried to debug the code of unmarshalling of List object result, and it seems the Unmarshaller expects the response to have a element called as KeyCount then only it sets it in the result object, otherwise it stays zero.
This is the actual log I get:
2019-02-02T15:04:19,789 ERROR [ZkCoordinator] org.apache.druid.segment.loading.SegmentLoaderLocalCacheManager - Failed to load segment in current location /data/projects/imply-2.8.9/var/druid/segment-cache, try next location if any: {class=org.apache.druid.segment.loading.SegmentLoaderLocalCacheManager, exceptionType=class org.apache.druid.segment.loading.SegmentLoadingException, exceptionMessage=Cannot find object for bucket[test4] and key[druid/segments/Consumer_Complaints/2016-01-01T00:00:00.000Z_2017-01-01T00:00:00.000Z/2019-02-02T15:03:18.693Z/0/index.zip], location=/data/projects/imply-2.8.9/var/druid/segment-cache}
org.apache.druid.segment.loading.SegmentLoadingException: Cannot find object for bucket[test4] and key[druid/segments/Consumer_Complaints/2016-01-01T00:00:00.000Z_2017-01-01T00:00:00.000Z/2019-02-02T15:03:18.693Z/0/index.zip]
at org.apache.druid.storage.s3.S3DataSegmentPuller.getSegmentFiles(S3DataSegmentPuller.java:139) ~[?:?]
at org.apache.druid.storage.s3.S3LoadSpec.loadSegment(S3LoadSpec.java:60) ~[?:?]
at org.apache.druid.segment.loading.SegmentLoaderLocalCacheManager.loadInLocation(SegmentLoaderLocalCacheManager.java:200) ~[druid-server-0.13.0-incubating-iap6-SNAPSHOT.jar:0.13.0-incubating-iap6-SNAPSHOT]
at org.apache.druid.segment.loading.SegmentLoaderLocalCacheManager.loadInLocationWithStartMarker(SegmentLoaderLocalCacheManager.java:188) ~[druid-server-0.13.0-incubating-iap6-SNAPSHOT.jar:0.13.0-incubating-iap6-SNAPSHOT]
at org.apache.druid.segment.loading.SegmentLoaderLocalCacheManager.loadSegmentWithRetry(SegmentLoaderLocalCacheManager.java:151) [
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org