Spark SQL - Reading HCatalog Table

[Sandip Mehta <sa...@gmail.com>]

Hi All,

I have a table created in Hive and stored/read using HCatalog. Table is in ORC format. I want to read this table in Spark SQL and do the join with RDDs.  How can i connect to HCatalog and get data from Spark SQL?

SM



---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

consumergroup not working

[Hudong Wang <ju...@hotmail.com>]

Hi, I am trying to read data from kafka in zookeeper mode with following code.
    val kafkaParams = Map[String, String] (      "zookeeper.connect" -> zookeeper,      "metadata.broker.list" -> brokers,      "group.id" -> consumerGroup,      "auto.offset.reset" -> autoOffsetReset    )
    return KafkaUtils.createStream[String, String, kafka.serializer.StringDecoder, kafka.serializer.StringDecoder](        ssc,        kafkaParams,        Map(topic -> 1),        StorageLevel.MEMORY_AND_DISK_SER_2)
I specified the group.id but in zookeeper I don't see my consumer group, though I see other people's consumer groups. 
Did I do something wrong in my code? Is there any way I can troubleshoot?
Thanks,Tony