Re: May I take this issue --hbase-spark

[nurseryboy <zg...@163.com>]

hi Ted 
   You are right, one sample is from Hive,  our hive version is 2.3.3. 
  The hive sample is just to show the create table syntax, it will not
impact the hbase-spark part. 
   
   The demo I am preparing that, thank you. 
  
Regards
Bill 


Ted Yu-3 wrote
> bq. ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe'
> 
> The above implies dependency on some class from Hive.
> 
> Which Hive release would you use if you choose the above route ?
> 
> Looking forward to your demo.
> 
> On Tue, Jul 31, 2018 at 9:09 AM bill.yunfu <

> guangcheng.zgc@

> >
> wrote:
> 
>> hi Ted
>>    Thank you for replying.
>> The sql support means user can directly use spark sql to create table and
>> query data from HBase. we found two sql support on HBase
>> SHC use following command to create table in spark sql:
>> CREATE TABLE spark_hbase USING
>> org.apache.spark.sql.execution.datasources.hbase
>>       OPTIONS ('catalog'=
>>       '{"table":{"namespace":"default", "name":"test",
>> "tableCoder":"PrimitiveType"},"rowkey":"key",
>>       "columns":{
>>       "col0":{"cf":"rowkey", "col":"key", "type":"string"},
>>       "col1":{"cf":"cf", "col":"a", "type":"string"}}}'
>>       )
>> (SHC is a project can get details from:
>> https://github.com/hortonworks-spark/shc)
>> In spark sql also can use hive command to create table:
>> create  table spark_hbase (col0 string, col1 string)
>>     ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe' with
>> SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:a")
>>     STORED AS
>>         INPUTFORMAT
>> 'org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat'
>>         OUTPUTFORMAT
>> 'org.apache.hadoop.hive.hbase.HiveHBaseTableOutputFormat'
>>                 tblproperties ("hbase.table.name" = "test");
>>
>> So we want make a similar DDL to create the table for hbase-spark model
>> and
>> query with the spark sql.
>>
>> And for the Spark release, we suggestion first target at spark 2.y, for
>> example the spark 2.2.2 which is stability now.
>>
>> We will create a demo base on hbase-spark model with sql support in
>> local,
>> then share here to discuss.
>>
>> Regards
>> Bill
>>
>>
>> Ted Yu-3 wrote
>> > For SQL support, can you be more specific on how the SQL support would
>> be
>> > added ?
>> >
>> > Maybe you can illustrate some examples showing the enhanced SQL syntax.
>> >
>> > Also, which Spark release(s) would be targeted?
>> >
>> > Thanks
>> >
>> > On Mon, Jul 30, 2018 at 10:57 AM bill.yunfu <
>>
>> > guangcheng.zgc@
>>
>> > >
>> > wrote:
>>
>>
>>
>>
>>
>> --
>> Sent from:
>> http://apache-hbase.679495.n3.nabble.com/HBase-Developer-f679493.html
>>





--
Sent from: http://apache-hbase.679495.n3.nabble.com/HBase-Developer-f679493.html