[GitHub] [incubator-druid] havannavar opened a new issue #6920: Kafka indexing service - unable to create datasource

[GitBox <gi...@apache.org>]

havannavar opened a new issue #6920: Kafka indexing service - unable to create datasource 
URL: https://github.com/apache/incubator-druid/issues/6920
 
 
   I'm using druid 0.13.0-incubating, Kafka Indexing service, amazon s3 and Postgres, Though topic is created in Kafka but unable to create datasource. No exception or Fails messages in the Logs of coordinator and overload and FAILED logs. Also, I can see records in druid_tasks, druid_supervisor tables and I don't see any records in **druid_datasource**. Can someone help me, going clueless on this issue.
   
   Below is my **common.runtime. properties**
   ````
   druid.extensions.loadList=[ "druid-kafka-indexing-service", "druid-s3-extensions", "postgresql-metadata-storage"]  
   
   #
   # Zookeeper
   #
   
   druid.zk.service.host=<ipaddress>
   druid.zk.paths.base=/druid 
   
   # For PostgreSQL (make sure to additionally include the Postgres extension):
   druid.metadata.storage.type=postgresql
   druid.metadata.storage.connector.connectURI=jdbc:postgresql://<ip>:<port>/druid
   druid.metadata.storage.connector.user=druid
   druid.metadata.storage.connector.password=<pwd> 
   
   #
   # Deep storage
   #
   # For S3:
   druid.storage.type=s3
   druid.storage.bucket= myBucket
   druid.storage.baseKey=druid/segments
   druid.s3.accessKey=key
   druid.s3.secretKey=secreteKey
   
   #
   # Indexing service logs
   #
   
   # For S3:
   druid.indexer.logs.type=s3
   druid.indexer.logs.s3Bucket=myBucket
   druid.indexer.logs.s3Prefix=druid/indexing-logs
   
   #
   # Service discovery
   #
   
   druid.selectors.indexing.serviceName=druid/overlord
   druid.selectors.coordinator.serviceName=druid/coordinator
   
   #
   # Monitoring
   #
   
   druid.monitoring.monitors=["org.apache.druid.java.util.metrics.JvmMonitor"]
   druid.emitter=logging
   druid.emitter.logging.logLevel=info
   
   # Storage type of double columns
   # ommiting this will lead to index double as float at the storage layer
   
   druid.indexing.doubleStorage=double
   ```
   `
   
   And Below is my datasource 
   
   ```
   {
     "type": "kafka",
     "dataSchema": {
       "dataSource": "4pm_27thjan",
       "parser": {
         "type": "string",
         "parseSpec": {
           "format": "json",
           "timestampSpec": {
             "column": "time",
             "format": "auto"
           },
           "dimensionsSpec": {
             "dimensions": [
               "channel",
               "cityName",
               "comment",
               "countryIsoCode",
               "countryName",
               "isAnonymous",
               "isMinor",
               "isNew",
               "isRobot",
               "isUnpatrolled",
               "metroCode",
               "namespace",
               "page",
               "regionIsoCode",
               "regionName",
               "user",
               { "name": "added", "type": "long" },
               { "name": "deleted", "type": "long" },
               { "name": "delta", "type": "long" }
             ]
           }
         }
       },
       "metricsSpec" : [],
       "granularitySpec": {
         "type": "uniform",
         "segmentGranularity": "HOUR",
         "queryGranularity": "NONE",
         "rollup": false
       }
     },
     "tuningConfig": {
       "type": "kafka",
       "reportParseExceptions": false
     },
     "ioConfig": {
       "topic": "4pm_27thjan_topic",
       "replicas": 1,
       "taskDuration": "PT1H",
       "completionTimeout": "PT2H",
       "consumerProperties": {
         "bootstrap.servers": "<broker1>:9092,<broker2>:9096,<broker3>:9010"
       }
     }
   }
   ```
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org