Re: [jira] [Commented] (KYLIN-679) Adding Spark Support to Apache Kylin

[Sarnath <st...@gmail.com>]

One can consider TEZ exec engine for hive. Spark exec engine for hive is
also under consideration as I see from hive site.
On May 31, 2015 11:34 PM, "Henry Saputra (JIRA)" <ji...@apache.org> wrote:

>
>     [
> https://issues.apache.org/jira/browse/KYLIN-679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14566629#comment-14566629
> ]
>
> Henry Saputra commented on KYLIN-679:
> -------------------------------------
>
> Integration should be done via abstraction to support different runtime
> engine, including native one from Kylin.
>
> Tight couple with Spark will make the project unable to leverage other
> runtime engines.
>
> > Adding Spark Support to Apache Kylin
> > ------------------------------------
> >
> >                 Key: KYLIN-679
> >                 URL: https://issues.apache.org/jira/browse/KYLIN-679
> >             Project: Kylin
> >          Issue Type: New Feature
> >          Components: General
> >            Reporter: Luke Han
> >             Fix For: Future
> >
> >
> > Challenges in current architecture:
> > High latency when reading data from Hive
> > --Several hours to fetch data when join big tables
> > --Route to SQL-on-Hadoop turned off due to performance issue
> > Time-to-Market of data latency
> > --Huge IO & Network traffic with MR jobs
> > Streaming
> > --Streaming process and pre-calculate cubes
> > Where Spark could bring benefits to Kylin:
> > Integrating with Spark SQL:
> > --Option I: Read data from SparkSQL instead of Hive
> > --Option II: Route unsupported queries to SparkSQL
> > --Option III: Kylin to be OLAP source of SparkSQL
> > Spark Cube Build Engine
> > --Efficiency cube generate engine with Spark
> > Spark Streaming
> > --Leverage SparkStreaming for StreamingOLAP
> > HBase?
> > --Any idea?
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>