You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by KylinPOC <sa...@gmail.com> on 2016/01/16 08:48:45 UTC
Support for Hive on Tez or Hive on Spark, cube build automation and
best practices
Support for Hive on Tez or Hive on Spark - Is this in works or available with
the existing release? If not, is it on the product roadmap and the duration
for its availability?
Does Kylin do automated cube build?
Is the cube available to the users during cube build? If not, how cube build
time is minimized for extremely large cubes?
What are the best practices for extremely large cube creation in Kylin?
--
View this message in context: http://apache-kylin.74782.x6.nabble.com/Support-for-Hive-on-Tez-or-Hive-on-Spark-cube-build-automation-and-best-practices-tp3280.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: Support for Hive on Tez or Hive on Spark, cube build automation
and best practices
Posted by Li Yang <li...@apache.org>.
In principal, Kylin does not do any scheduling stuff. Because only upstream
ETL knows when the data lands in hive. That's why Kylin provides Rest API
for upstream to call when data is ready.
On Sat, Jan 16, 2016 at 8:18 PM, hongbin ma <ma...@apache.org> wrote:
> kylin invokes shell to execute hive commands, so I think Hive on Tez or
> Hive on Spark is transparent to Kylin, have you ever tried out before
> asking? If there's known issue here please open a ticket, the community
> would love to help.
>
> Kylin does not provided complicated scheduling for the moment. However we
> do plan to do something like
> https://issues.apache.org/jira/browse/KYLIN-1249, although it may still
> require external scheduling systems like crond.
>
> Cube consists of segments, cubing job is essentially appending segments to
> it(or refreshing existing segments). Cubing job will not affect cube
> segments util the job is done.
>
>
>
>
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
>
Re: Support for Hive on Tez or Hive on Spark, cube build automation
and best practices
Posted by hongbin ma <ma...@apache.org>.
kylin invokes shell to execute hive commands, so I think Hive on Tez or
Hive on Spark is transparent to Kylin, have you ever tried out before
asking? If there's known issue here please open a ticket, the community
would love to help.
Kylin does not provided complicated scheduling for the moment. However we
do plan to do something like
https://issues.apache.org/jira/browse/KYLIN-1249, although it may still
require external scheduling systems like crond.
Cube consists of segments, cubing job is essentially appending segments to
it(or refreshing existing segments). Cubing job will not affect cube
segments util the job is done.
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone