You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@ignite.apache.org by lk_hadoop <lk...@163.com> on 2018/11/23 06:56:54 UTC

can ignite accelerate sparkOnHive

hi all,
    I think use hive as DW and use spark do some OLAP on hive is quite common . So ,can ignite accelerate sparkOnHive? and is there something special to do with this , or all what I should do is just make HDFS as IGFS SecondaryFileSystem ? 

2018-11-23


lk_hadoop

Re: Re: can ignite accelerate sparkOnHive

Posted by lk_hadoop <lk...@163.com>.

@Franke  , we use spark sql do OLAP and ETL , where the data was store on hive. So I wonder if ignite is suitable for such a scenario, make IFGS as a transparent cache about HDFS , and spark sql on hive can benefit from it.
2018-11-26 

lk_hadoop 



发件人：Jörn Franke <jo...@gmail.com>
发送时间：2018-11-23 16:12
主题：Re: can ignite accelerate sparkOnHive
收件人："user"<us...@ignite.apache.org>
抄送：

I think the more important question is why you need this. There are many different ways on accelerating warehouse depending on what you want to achieve.

Am 23.11.2018 um 07:56 schrieb lk_hadoop <lk...@163.com>:


hi all,
    I think use hive as DW and use spark do some OLAP on hive is quite common . So ,can ignite accelerate sparkOnHive? and is there something special to do with this , or all what I should do is just make HDFS as IGFS SecondaryFileSystem ? 

2018-11-23


lk_hadoop

Re: can ignite accelerate sparkOnHive

Posted by Jörn Franke <jo...@gmail.com>.

I think the more important question is why you need this. There are many different ways on accelerating warehouse depending on what you want to achieve.

> Am 23.11.2018 um 07:56 schrieb lk_hadoop <lk...@163.com>:
> 
> hi all,
>     I think use hive as DW and use spark do some OLAP on hive is quite common . So ,can ignite accelerate sparkOnHive? and is there something special to do with this , or all what I should do is just make HDFS as IGFS SecondaryFileSystem ? 
>  
> 2018-11-23
> lk_hadoop