You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/09/14 03:50:12 UTC
[GitHub] [druid] tarunparackal opened a new issue #10393: Error while adding a global cached lookup with has header and skip header for csv file
tarunparackal opened a new issue #10393:
URL: https://github.com/apache/druid/issues/10393
While adding a global cached lookup with has header and skip header for csv file gives the following error.
`java.lang.UnsupportedOperationException: hasHeaderRow or maxSkipHeaderRows is not supported. Please check the indexTask supports these options.
`
### Affected Version
All versions
### Description
Please include as much detailed information about the problem as possible.
- Configurations in use : Selected the following value from the UI
Globally cached lookup type :uri
URI prefix : file:/Users/tarun/Desktop/tsv/channel_lookup_v1.csv
Format : csv
Columns : id, type
Key column :id
Value column : type
Has header row : True
Skip header rows : 3
Injective : true
- Steps to reproduce the problem
Create the lookup using ui . I have used the following csv file with first 3 rows as blank
```
id,type
8009981242,NOT_SET
9309242846,NOT_SET
```
- The error message or stack traces encountered. Providing more context, such as nearby log messages or even entire logs, can be helpful.
```
2020-09-09T08:12:35,762 ERROR [NamespaceExtractionCacheManager-1] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Failed to update namespace [UriExtractionNamespace{uri=null, uriPrefix=s3://imply-cloud-support-data/3ded587e-86bd-4b5b-bd7c-1d2c53ec98df/lookup/, namespaceParseSpec=CSVFlatDataParser{columns=[campaignid, googleadsadvertisingchanneltype], keyColumn='campaignid', valueColumn='googleadsadvertisingchanneltype'}, fileRegex='channel_lookup_v1.csv', pollPeriod=PT5M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@7353553c
java.lang.UnsupportedOperationException: hasHeaderRow or maxSkipHeaderRows is not supported. Please check the indexTask supports these options.
at org.apache.druid.java.util.common.parsers.AbstractFlatTextFormatParser.parseToMap(AbstractFlatTextFormatParser.java:121) ~[druid-core-0.19.0-iap5.jar:0.19.0-iap5]
at org.apache.druid.query.lookup.namespace.UriExtractionNamespace$DelegateParser.parseToMap(UriExtractionNamespace.java:218) ~[?:?]
at org.apache.druid.data.input.MapPopulator$1.processLine(MapPopulator.java:94) ~[?:?]
at com.google.common.io.CharStreams.readLines(CharStreams.java:317) ~[guava-16.0.1.jar:?]
at com.google.common.io.CharSource.readLines(CharSource.java:239) ~[guava-16.0.1.jar:?]
at org.apache.druid.data.input.MapPopulator.populate(MapPopulator.java:82) ~[?:?]
at org.apache.druid.server.lookup.namespace.UriCacheGenerator.lambda$generateCache$0(UriCacheGenerator.java:149) ~[?:?]
at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:87) ~[druid-core-0.19.0-iap5.jar:0.19.0-iap5]
at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:115) ~[druid-core-0.19.0-iap5.jar:0.19.0-iap5]
at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:105) ~[druid-core-0.19.0-iap5.jar:0.19.0-iap5]
at org.apache.druid.server.lookup.namespace.UriCacheGenerator.generateCache(UriCacheGenerator.java:116) ~[druid-lookups-cached-global-0.19.0-iap5.jar:0.19.0-iap5]
at org.apache.druid.server.lookup.namespace.UriCacheGenerator.generateCache(UriCacheGenerator.java:47) ~[druid-lookups-cached-global-0.19.0-iap5.jar:0.19.0-iap5]
at org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl.tryUpdateCache(CacheScheduler.java:229) [druid-lookups-cached-global-0.19.0-iap5.jar:0.19.0-iap5]
at org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl.updateCache(CacheScheduler.java:208) [druid-lookups-cached-global-0.19.0-iap5.jar:0.19.0-iap5]
at org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl.access$600(CacheScheduler.java:144) [druid-lookups-cached-global-0.19.0-iap5.jar:0.19.0-iap5]
at org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl$2.run(CacheScheduler.java:190) [druid-lookups-cached-global-0.19.0-iap5.jar:0.19.0-iap5]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_262]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_262]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_262]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_262]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_262]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_262]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_262]
2020-09-09T08:16:35,696 INFO [LookupExtractorFactoryContainerProvider-MainThread] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Closing namespace [UriExtractionNamespace{uri=null, uriPrefix=s3://imply-cloud-support-data/3ded587e-86bd-4b5b-bd7c-1d2c53ec98df/lookup/, namespaceParseSpec=CSVFlatDataParser{columns=[campaignid, googleadsadvertisingchanneltype], keyColumn='campaignid', valueColumn='googleadsadvertisingchanneltype'}, fileRegex='channel_lookup_v1.csv', pollPeriod=PT5M}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@7353553c
2020-09-09T08:16:35,815 INFO [NamespaceExtractionCacheManager-1]
```
- Any debugging that you have already done
While debugging i relaized the error is due to startFileFromBeginning method of AbstractFlatTextformatParser is not called.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s closed issue #10393: Error while adding a global cached lookup with has header and skip header for csv file
Posted by GitBox <gi...@apache.org>.
suneet-s closed issue #10393:
URL: https://github.com/apache/druid/issues/10393
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s closed issue #10393: Error while adding a global cached lookup with has header and skip header for csv file
Posted by GitBox <gi...@apache.org>.
suneet-s closed issue #10393:
URL: https://github.com/apache/druid/issues/10393
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s closed issue #10393: Error while adding a global cached lookup with has header and skip header for csv file
Posted by GitBox <gi...@apache.org>.
suneet-s closed issue #10393:
URL: https://github.com/apache/druid/issues/10393
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org