You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by Sarnath K <st...@gmail.com> on 2016/08/25 06:00:17 UTC

Hive server vs Impala server

Hi,
I tried searching for this a lot but could find a convincing answer.

Can Kylin use Impala JDBC driver underneath to run queries on Hive table
instead of Hive server?

If this is not possible, I would like to understand why this is not
possible and if there is a roadmap for this.

Thanks,
Best,
Sarnath

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
I tailed the server log(Kylin.out) and it did not show any exception during
this process. (Is that because kylin catches those exceptions)
I am sure there are no cubes built with a Where condition. How can this be
explained? Any ideas?

On Sep 9, 2016 10:51, "ShaoFeng Shi" <sh...@apache.org> wrote:

> Sarnath,  please check the server side error to see why it wasn't
> redirected (the rediection only happens when the exception
> is NoRealizationFoundException).
>
> 2016-09-08 14:17 GMT+08:00 Sarnath K <st...@gmail.com>:
>
> > Hi,
> >
> > Thanks for the info. I am trying to execute the following SQL query from
> a
> > Java app using Kylin JDBC driver.  The same query works fine in Impala
> and
> > am able to catch the exception below and redirect to Impala successfully
> > from the Java app.
> >
> > Kylin is on Ubuntu 12.04 64bit Linux machine if that helps. HBase is CdH
> > 0.98.6. Kylin is 1.5.3 pre-built binary.
> > Java app is run from Windows machine from eclipse. It is using the right
> > Kylin JDBC 1.5.3 drivers to connect.
> >
> > Query: select productid, sum(quantity) from hive_aggregator_kylin where
> > quantity < 100 group by productid;
> >
> >
> >
> > Error Trace:
> >
> >
> >
> > *java.sql.SQLException*: Error while executing SQL "select productid,
> > sum(quantity) from hive_aggregator_kylin where quantity < 100 group by
> > productid": *java.sql.SQLException*:*java.io.IOException*: POST failed,
> > error code 500 and response: {"url":"http://
> > <http://10.139.9.238:7070/kylin/api/query>hostip
> > <http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
> > <http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
> > executing SQL \"select productid, sum(quantity) from
> hive_aggregator_kylin
> > where quantity < 100 group by productid\": null"}
> >
> >        at
> > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.
> > Helper.createException(
> > *Helper.java:56*)
> >
> >        at
> > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.
> > Helper.createException(
> > *Helper.java:41*)
> >
> >        at
> > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.
> AvaticaStatement.
> > executeInternal(
> > *AvaticaStatement.java:143*)
> >
> >        at
> > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.
> > AvaticaStatement.executeQuery(
> > *AvaticaStatement.java:186*)
> >
> >        at Kylin_Impala_JDBC.main(*Kylin_Impala_JDBC.java:27*)
> >
> > Caused by: *java.lang.RuntimeException*:*java.sql.SQLException*:
> > *java.io.IOException*: POST failed, error code 500 and response: {"url":"
> > http:// <http://10.139.9.238:7070/kylin/api/query>hostip
> > <http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
> > <http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
> > executing SQL \"select productid, sum(quantity) from
> hive_aggregator_kylin
> > where quantity < 100 group by productid\": null"}
> >
> >        at org.apache.kylin.jdbc.KylinMeta.prepareAndExecute(
> > *KylinMeta.java:81*)
> >
> >        at
> > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.
> AvaticaConnection.
> > prepareAndExecuteInternal(
> > *AvaticaConnection.java:571*)
> >
> >        at
> > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.
> AvaticaStatement.
> > executeInternal(
> > *AvaticaStatement.java:135*)
> >
> >        ... 2 more
> >
> > Caused by: *java.sql.SQLException*:*java.io.IOException*: POST failed,
> > error code 500 and response: {"url":"http://
> > <http://10.139.9.238:7070/kylin/api/query>hostip
> > <http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
> > <http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
> > executing SQL \"select productid, sum(quantity) from
> hive_aggregator_kylin
> > where quantity < 100 group by productid\": null"}
> >
> >        at org.apache.kylin.jdbc.KylinResultSet.execute(
> > *KylinResultSet.java:64*)
> >
> >        at
> > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.
> > AvaticaConnection$1.execute(
> > *AvaticaConnection.java:566*)
> >
> >        at org.apache.kylin.jdbc.KylinMeta.prepareAndExecute(
> > *KylinMeta.java:77*)
> >
> >        ... 4 more
> >
> > Caused by: *java.io.IOException*: POST failed, error code 500 and
> response:
> > {"url":"http:// <http://10.139.9.238:7070/kylin/api/query>hostip
> > <http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
> > <http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
> > executing SQL \"select productid, sum(quantity) from
> hive_aggregator_kylin
> > where quantity < 100 group by productid\": null"}
> >
> >        at org.apache.kylin.jdbc.KylinClient.asIOException(
> > *KylinClient.java:414*)
> >
> >        at org.apache.kylin.jdbc.KylinClient.executeKylinQuery(
> > *KylinClient.java:374*)
> >
> >        at org.apache.kylin.jdbc.KylinClient.executeQuery(
> > *KylinClient.java:325*)
> >
> >        at org.apache.kylin.jdbc.KylinResultSet.execute(
> > *KylinResultSet.java:62*)
> >
> >        ... 6 more
> >
> > Exception in thread "main"*java.lang.NullPointerException*
> >
> >        at Kylin_Impala_JDBC.main(*Kylin_Impala_JDBC.java:36*)
> >
> >
> > Thanks
> > Best
> > Sarnath
> >
> > On Sep 7, 2016 8:55 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:
> >
> > > 1. Could you please provide the error trace?
> > >
> > > 3. From a recent version Kylin start to support "select *" with the
> > > aggregated data in cube.
> > >
> > > 2016-09-07 21:40 GMT+08:00 Sarnath K <st...@gmail.com>:
> > >
> > > > Hi,
> > > > Today I tested this routing feature. It did not work as expected.
> > > >
> > > > 1. When I execute Group By queries along with WHERE conditions on a
> > plain
> > > > cube  built without conditions,  the JDBC driver simply throws an
> > > > exception.  I was expecting Kylin to redirect to impala which I had
> > > > configured through hive.url configuration parameter. This did not
> > > happen. I
> > > > see the code that tries olapenumerator and if that catches exception,
> > > > redirects to Hive enumerator. But the output I see does not
> correspond
> > to
> > > > the code. What am I missing?
> > > >
> > > > 2. For the scenario described above,  I am able to catch the
> exception
> > in
> > > > Java app code and redirect to Impala. This works like a charm and is
> > very
> > > > fast. Not sure if self-service BI tools will do all these. Consider
> > > > filtering a pivot chart.  This translates to a SQL with a WHERE
> > followed
> > > by
> > > > a GROUP BY. This is not a rare case but rather commonly occurring
> case.
> > > >
> > > > 3. When I issue 'select * from table',  Kylin returns cube data
> instead
> > > of
> > > > giving raw table contents.  This is a bit unexpected.  But I remember
> > > > reading somewhere in the forums earlier that this is actually a
> > feature.
> > > > Does my memory serve me right?
> > > >
> > > > Thanks,
> > > > Best,
> > > > Sarnath
> > > > Sent from 1+3
> > > >
> > > > On Sep 2, 2016 9:37 AM, "Sarnath K" <st...@gmail.com> wrote:
> > > >
> > > > > Sure. I assume hive routing is enabled by default so that I should
> be
> > > > able
> > > > > to do 'select *' queries through JDBC. This is what we have been
> > trying
> > > > > with 1.1 and t did not work. The webapp does not use routing and it
> > is
> > > > > documented. No worries on the webapp part.
> > > > >
> > > > > Is there any docs on uninstalling Kylin -remove all HBase tables,
> > > > > Meta-Data and wipe of all intermediate hive tables etc..?
> > > > >
> > > > > Also, I am using prebuilt binaries for HBase 0.98.
> > > > > We are using HBase 0.98.6 that comes with CDH.. Hope this config
> > works.
> > > > > On Sep 2, 2016 6:43 AM, "ShaoFeng Shi" <sh...@apache.org>
> > wrote:
> > > > >
> > > > >> Hi Sarnath,
> > > > >>
> > > > >> For kylin.sh on Ubuntu issue, there is an open JIRA (
> > > > >> https://issues.apache.org/jira/browse/KYLIN-1978), would you like
> > to
> > > > >> contribute your finding as a patch? thank you!
> > > > >>
> > > > >> The "kylin.route.hive.enabled" isn't in use anymore; what need is
> > just
> > > > >> "hive.url", "hive.user" and "hive.password".
> > > > >>
> > > > >> All the config parameters can be found in the java
> > > > >> class org.apache.kylin.common.KylinConfigBase.java; There is a
> work
> > > > item
> > > > >> about adding doc for these parameters, we will add it soon.
> > > > >>
> > > > >> Changing kylin.properties need restart kylin server to take
> > effective;
> > > > >> Besides I havn't heard such problem (even with 1.1), please retry
> > and
> > > > >> report problem with detail information and we can have a check.
> > > > >>
> > > > >>
> > > > >> 2016-09-01 23:18 GMT+08:00 Sarnath K <st...@gmail.com>:
> > > > >>
> > > > >> > I just downloaded 1.5.3 version of Kylin. The startup scripts
> fail
> > > in
> > > > >> > Ubuntu because of "sh" in place of "bash". Replaced that in 2
> > > scripts
> > > > to
> > > > >> > get going.
> > > > >> >
> > > > >> > And, apart from hive.URL and other variables, should I also set
> > > > >> > kylin.route.hive.enabled to true to enable routing to Hive?
> > > > >> >
> > > > >> > Where can I find a list of all config vars? Looks like the
> > > > >> documentation is
> > > > >> > very sparse.
> > > > >> >
> > > > >> > Also, the webapp does not reflect kylin.properties well. Even
> > after
> > > > >> > refreshes.... At least this was the case with 1.1incubating.
> > > > >> > On Aug 30, 2016 12:40 PM, "ShaoFeng Shi" <
> shaofengshi@apache.org>
> > > > >> wrote:
> > > > >> >
> > > > >> > > Hi Sarnath, I totally agree with you; we're looking forward to
> > see
> > > > >> your
> > > > >> > > sharing and patch.
> > > > >> > >
> > > > >> > > 2016-08-30 13:47 GMT+08:00 Sarnath K <st...@gmail.com>:
> > > > >> > >
> > > > >> > > > Sure. I just hope this never goes away. It is a great
> feature
> > > and
> > > > BI
> > > > >> > > tools
> > > > >> > > > like Tableau can work seamlessly. And, it is a point I keep
> > > > telling
> > > > >> > > > customers too. It is a great optimization that is really
> > > awesome.
> > > > >> > > > I will test it with Impala and post back.
> > > > >> > > >
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> > > --
> > > > >> > > Best regards,
> > > > >> > >
> > > > >> > > Shaofeng Shi
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Best regards,
> > > > >>
> > > > >> Shaofeng Shi
> > > > >>
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > Shaofeng Shi
> > >
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>

Re: Hive server vs Impala server

Posted by ShaoFeng Shi <sh...@apache.org>.
Sarnath,  please check the server side error to see why it wasn't
redirected (the rediection only happens when the exception
is NoRealizationFoundException).

2016-09-08 14:17 GMT+08:00 Sarnath K <st...@gmail.com>:

> Hi,
>
> Thanks for the info. I am trying to execute the following SQL query from a
> Java app using Kylin JDBC driver.  The same query works fine in Impala and
> am able to catch the exception below and redirect to Impala successfully
> from the Java app.
>
> Kylin is on Ubuntu 12.04 64bit Linux machine if that helps. HBase is CdH
> 0.98.6. Kylin is 1.5.3 pre-built binary.
> Java app is run from Windows machine from eclipse. It is using the right
> Kylin JDBC 1.5.3 drivers to connect.
>
> Query: select productid, sum(quantity) from hive_aggregator_kylin where
> quantity < 100 group by productid;
>
>
>
> Error Trace:
>
>
>
> *java.sql.SQLException*: Error while executing SQL "select productid,
> sum(quantity) from hive_aggregator_kylin where quantity < 100 group by
> productid": *java.sql.SQLException*:*java.io.IOException*: POST failed,
> error code 500 and response: {"url":"http://
> <http://10.139.9.238:7070/kylin/api/query>hostip
> <http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
> <http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
> executing SQL \"select productid, sum(quantity) from hive_aggregator_kylin
> where quantity < 100 group by productid\": null"}
>
>        at
> org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.
> Helper.createException(
> *Helper.java:56*)
>
>        at
> org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.
> Helper.createException(
> *Helper.java:41*)
>
>        at
> org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaStatement.
> executeInternal(
> *AvaticaStatement.java:143*)
>
>        at
> org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.
> AvaticaStatement.executeQuery(
> *AvaticaStatement.java:186*)
>
>        at Kylin_Impala_JDBC.main(*Kylin_Impala_JDBC.java:27*)
>
> Caused by: *java.lang.RuntimeException*:*java.sql.SQLException*:
> *java.io.IOException*: POST failed, error code 500 and response: {"url":"
> http:// <http://10.139.9.238:7070/kylin/api/query>hostip
> <http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
> <http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
> executing SQL \"select productid, sum(quantity) from hive_aggregator_kylin
> where quantity < 100 group by productid\": null"}
>
>        at org.apache.kylin.jdbc.KylinMeta.prepareAndExecute(
> *KylinMeta.java:81*)
>
>        at
> org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaConnection.
> prepareAndExecuteInternal(
> *AvaticaConnection.java:571*)
>
>        at
> org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaStatement.
> executeInternal(
> *AvaticaStatement.java:135*)
>
>        ... 2 more
>
> Caused by: *java.sql.SQLException*:*java.io.IOException*: POST failed,
> error code 500 and response: {"url":"http://
> <http://10.139.9.238:7070/kylin/api/query>hostip
> <http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
> <http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
> executing SQL \"select productid, sum(quantity) from hive_aggregator_kylin
> where quantity < 100 group by productid\": null"}
>
>        at org.apache.kylin.jdbc.KylinResultSet.execute(
> *KylinResultSet.java:64*)
>
>        at
> org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.
> AvaticaConnection$1.execute(
> *AvaticaConnection.java:566*)
>
>        at org.apache.kylin.jdbc.KylinMeta.prepareAndExecute(
> *KylinMeta.java:77*)
>
>        ... 4 more
>
> Caused by: *java.io.IOException*: POST failed, error code 500 and response:
> {"url":"http:// <http://10.139.9.238:7070/kylin/api/query>hostip
> <http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
> <http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
> executing SQL \"select productid, sum(quantity) from hive_aggregator_kylin
> where quantity < 100 group by productid\": null"}
>
>        at org.apache.kylin.jdbc.KylinClient.asIOException(
> *KylinClient.java:414*)
>
>        at org.apache.kylin.jdbc.KylinClient.executeKylinQuery(
> *KylinClient.java:374*)
>
>        at org.apache.kylin.jdbc.KylinClient.executeQuery(
> *KylinClient.java:325*)
>
>        at org.apache.kylin.jdbc.KylinResultSet.execute(
> *KylinResultSet.java:62*)
>
>        ... 6 more
>
> Exception in thread "main"*java.lang.NullPointerException*
>
>        at Kylin_Impala_JDBC.main(*Kylin_Impala_JDBC.java:36*)
>
>
> Thanks
> Best
> Sarnath
>
> On Sep 7, 2016 8:55 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:
>
> > 1. Could you please provide the error trace?
> >
> > 3. From a recent version Kylin start to support "select *" with the
> > aggregated data in cube.
> >
> > 2016-09-07 21:40 GMT+08:00 Sarnath K <st...@gmail.com>:
> >
> > > Hi,
> > > Today I tested this routing feature. It did not work as expected.
> > >
> > > 1. When I execute Group By queries along with WHERE conditions on a
> plain
> > > cube  built without conditions,  the JDBC driver simply throws an
> > > exception.  I was expecting Kylin to redirect to impala which I had
> > > configured through hive.url configuration parameter. This did not
> > happen. I
> > > see the code that tries olapenumerator and if that catches exception,
> > > redirects to Hive enumerator. But the output I see does not correspond
> to
> > > the code. What am I missing?
> > >
> > > 2. For the scenario described above,  I am able to catch the exception
> in
> > > Java app code and redirect to Impala. This works like a charm and is
> very
> > > fast. Not sure if self-service BI tools will do all these. Consider
> > > filtering a pivot chart.  This translates to a SQL with a WHERE
> followed
> > by
> > > a GROUP BY. This is not a rare case but rather commonly occurring case.
> > >
> > > 3. When I issue 'select * from table',  Kylin returns cube data instead
> > of
> > > giving raw table contents.  This is a bit unexpected.  But I remember
> > > reading somewhere in the forums earlier that this is actually a
> feature.
> > > Does my memory serve me right?
> > >
> > > Thanks,
> > > Best,
> > > Sarnath
> > > Sent from 1+3
> > >
> > > On Sep 2, 2016 9:37 AM, "Sarnath K" <st...@gmail.com> wrote:
> > >
> > > > Sure. I assume hive routing is enabled by default so that I should be
> > > able
> > > > to do 'select *' queries through JDBC. This is what we have been
> trying
> > > > with 1.1 and t did not work. The webapp does not use routing and it
> is
> > > > documented. No worries on the webapp part.
> > > >
> > > > Is there any docs on uninstalling Kylin -remove all HBase tables,
> > > > Meta-Data and wipe of all intermediate hive tables etc..?
> > > >
> > > > Also, I am using prebuilt binaries for HBase 0.98.
> > > > We are using HBase 0.98.6 that comes with CDH.. Hope this config
> works.
> > > > On Sep 2, 2016 6:43 AM, "ShaoFeng Shi" <sh...@apache.org>
> wrote:
> > > >
> > > >> Hi Sarnath,
> > > >>
> > > >> For kylin.sh on Ubuntu issue, there is an open JIRA (
> > > >> https://issues.apache.org/jira/browse/KYLIN-1978), would you like
> to
> > > >> contribute your finding as a patch? thank you!
> > > >>
> > > >> The "kylin.route.hive.enabled" isn't in use anymore; what need is
> just
> > > >> "hive.url", "hive.user" and "hive.password".
> > > >>
> > > >> All the config parameters can be found in the java
> > > >> class org.apache.kylin.common.KylinConfigBase.java; There is a work
> > > item
> > > >> about adding doc for these parameters, we will add it soon.
> > > >>
> > > >> Changing kylin.properties need restart kylin server to take
> effective;
> > > >> Besides I havn't heard such problem (even with 1.1), please retry
> and
> > > >> report problem with detail information and we can have a check.
> > > >>
> > > >>
> > > >> 2016-09-01 23:18 GMT+08:00 Sarnath K <st...@gmail.com>:
> > > >>
> > > >> > I just downloaded 1.5.3 version of Kylin. The startup scripts fail
> > in
> > > >> > Ubuntu because of "sh" in place of "bash". Replaced that in 2
> > scripts
> > > to
> > > >> > get going.
> > > >> >
> > > >> > And, apart from hive.URL and other variables, should I also set
> > > >> > kylin.route.hive.enabled to true to enable routing to Hive?
> > > >> >
> > > >> > Where can I find a list of all config vars? Looks like the
> > > >> documentation is
> > > >> > very sparse.
> > > >> >
> > > >> > Also, the webapp does not reflect kylin.properties well. Even
> after
> > > >> > refreshes.... At least this was the case with 1.1incubating.
> > > >> > On Aug 30, 2016 12:40 PM, "ShaoFeng Shi" <sh...@apache.org>
> > > >> wrote:
> > > >> >
> > > >> > > Hi Sarnath, I totally agree with you; we're looking forward to
> see
> > > >> your
> > > >> > > sharing and patch.
> > > >> > >
> > > >> > > 2016-08-30 13:47 GMT+08:00 Sarnath K <st...@gmail.com>:
> > > >> > >
> > > >> > > > Sure. I just hope this never goes away. It is a great feature
> > and
> > > BI
> > > >> > > tools
> > > >> > > > like Tableau can work seamlessly. And, it is a point I keep
> > > telling
> > > >> > > > customers too. It is a great optimization that is really
> > awesome.
> > > >> > > > I will test it with Impala and post back.
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > --
> > > >> > > Best regards,
> > > >> > >
> > > >> > > Shaofeng Shi
> > > >> > >
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Best regards,
> > > >>
> > > >> Shaofeng Shi
> > > >>
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> > Shaofeng Shi
> >
>



-- 
Best regards,

Shaofeng Shi

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
Hi,

Thanks for the info. I am trying to execute the following SQL query from a
Java app using Kylin JDBC driver.  The same query works fine in Impala and
am able to catch the exception below and redirect to Impala successfully
from the Java app.

Kylin is on Ubuntu 12.04 64bit Linux machine if that helps. HBase is CdH
0.98.6. Kylin is 1.5.3 pre-built binary.
Java app is run from Windows machine from eclipse. It is using the right
Kylin JDBC 1.5.3 drivers to connect.

Query: select productid, sum(quantity) from hive_aggregator_kylin where
quantity < 100 group by productid;



Error Trace:



*java.sql.SQLException*: Error while executing SQL "select productid,
sum(quantity) from hive_aggregator_kylin where quantity < 100 group by
productid": *java.sql.SQLException*:*java.io.IOException*: POST failed,
error code 500 and response: {"url":"http://
<http://10.139.9.238:7070/kylin/api/query>hostip
<http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
<http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
executing SQL \"select productid, sum(quantity) from hive_aggregator_kylin
where quantity < 100 group by productid\": null"}

       at
org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.Helper.createException(
*Helper.java:56*)

       at
org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.Helper.createException(
*Helper.java:41*)

       at
org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaStatement.executeInternal(
*AvaticaStatement.java:143*)

       at
org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaStatement.executeQuery(
*AvaticaStatement.java:186*)

       at Kylin_Impala_JDBC.main(*Kylin_Impala_JDBC.java:27*)

Caused by: *java.lang.RuntimeException*:*java.sql.SQLException*:
*java.io.IOException*: POST failed, error code 500 and response: {"url":"
http:// <http://10.139.9.238:7070/kylin/api/query>hostip
<http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
<http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
executing SQL \"select productid, sum(quantity) from hive_aggregator_kylin
where quantity < 100 group by productid\": null"}

       at org.apache.kylin.jdbc.KylinMeta.prepareAndExecute(
*KylinMeta.java:81*)

       at
org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(
*AvaticaConnection.java:571*)

       at
org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaStatement.executeInternal(
*AvaticaStatement.java:135*)

       ... 2 more

Caused by: *java.sql.SQLException*:*java.io.IOException*: POST failed,
error code 500 and response: {"url":"http://
<http://10.139.9.238:7070/kylin/api/query>hostip
<http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
<http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
executing SQL \"select productid, sum(quantity) from hive_aggregator_kylin
where quantity < 100 group by productid\": null"}

       at org.apache.kylin.jdbc.KylinResultSet.execute(
*KylinResultSet.java:64*)

       at
org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaConnection$1.execute(
*AvaticaConnection.java:566*)

       at org.apache.kylin.jdbc.KylinMeta.prepareAndExecute(
*KylinMeta.java:77*)

       ... 4 more

Caused by: *java.io.IOException*: POST failed, error code 500 and response:
{"url":"http:// <http://10.139.9.238:7070/kylin/api/query>hostip
<http://10.139.9.238:7070/kylin/api/query>:7070/kylin/api/query
<http://10.139.9.238:7070/kylin/api/query>","exception":"Error while
executing SQL \"select productid, sum(quantity) from hive_aggregator_kylin
where quantity < 100 group by productid\": null"}

       at org.apache.kylin.jdbc.KylinClient.asIOException(
*KylinClient.java:414*)

       at org.apache.kylin.jdbc.KylinClient.executeKylinQuery(
*KylinClient.java:374*)

       at org.apache.kylin.jdbc.KylinClient.executeQuery(
*KylinClient.java:325*)

       at org.apache.kylin.jdbc.KylinResultSet.execute(
*KylinResultSet.java:62*)

       ... 6 more

Exception in thread "main"*java.lang.NullPointerException*

       at Kylin_Impala_JDBC.main(*Kylin_Impala_JDBC.java:36*)


Thanks
Best
Sarnath

On Sep 7, 2016 8:55 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:

> 1. Could you please provide the error trace?
>
> 3. From a recent version Kylin start to support "select *" with the
> aggregated data in cube.
>
> 2016-09-07 21:40 GMT+08:00 Sarnath K <st...@gmail.com>:
>
> > Hi,
> > Today I tested this routing feature. It did not work as expected.
> >
> > 1. When I execute Group By queries along with WHERE conditions on a plain
> > cube  built without conditions,  the JDBC driver simply throws an
> > exception.  I was expecting Kylin to redirect to impala which I had
> > configured through hive.url configuration parameter. This did not
> happen. I
> > see the code that tries olapenumerator and if that catches exception,
> > redirects to Hive enumerator. But the output I see does not correspond to
> > the code. What am I missing?
> >
> > 2. For the scenario described above,  I am able to catch the exception in
> > Java app code and redirect to Impala. This works like a charm and is very
> > fast. Not sure if self-service BI tools will do all these. Consider
> > filtering a pivot chart.  This translates to a SQL with a WHERE followed
> by
> > a GROUP BY. This is not a rare case but rather commonly occurring case.
> >
> > 3. When I issue 'select * from table',  Kylin returns cube data instead
> of
> > giving raw table contents.  This is a bit unexpected.  But I remember
> > reading somewhere in the forums earlier that this is actually a feature.
> > Does my memory serve me right?
> >
> > Thanks,
> > Best,
> > Sarnath
> > Sent from 1+3
> >
> > On Sep 2, 2016 9:37 AM, "Sarnath K" <st...@gmail.com> wrote:
> >
> > > Sure. I assume hive routing is enabled by default so that I should be
> > able
> > > to do 'select *' queries through JDBC. This is what we have been trying
> > > with 1.1 and t did not work. The webapp does not use routing and it is
> > > documented. No worries on the webapp part.
> > >
> > > Is there any docs on uninstalling Kylin -remove all HBase tables,
> > > Meta-Data and wipe of all intermediate hive tables etc..?
> > >
> > > Also, I am using prebuilt binaries for HBase 0.98.
> > > We are using HBase 0.98.6 that comes with CDH.. Hope this config works.
> > > On Sep 2, 2016 6:43 AM, "ShaoFeng Shi" <sh...@apache.org> wrote:
> > >
> > >> Hi Sarnath,
> > >>
> > >> For kylin.sh on Ubuntu issue, there is an open JIRA (
> > >> https://issues.apache.org/jira/browse/KYLIN-1978), would you like to
> > >> contribute your finding as a patch? thank you!
> > >>
> > >> The "kylin.route.hive.enabled" isn't in use anymore; what need is just
> > >> "hive.url", "hive.user" and "hive.password".
> > >>
> > >> All the config parameters can be found in the java
> > >> class org.apache.kylin.common.KylinConfigBase.java; There is a work
> > item
> > >> about adding doc for these parameters, we will add it soon.
> > >>
> > >> Changing kylin.properties need restart kylin server to take effective;
> > >> Besides I havn't heard such problem (even with 1.1), please retry and
> > >> report problem with detail information and we can have a check.
> > >>
> > >>
> > >> 2016-09-01 23:18 GMT+08:00 Sarnath K <st...@gmail.com>:
> > >>
> > >> > I just downloaded 1.5.3 version of Kylin. The startup scripts fail
> in
> > >> > Ubuntu because of "sh" in place of "bash". Replaced that in 2
> scripts
> > to
> > >> > get going.
> > >> >
> > >> > And, apart from hive.URL and other variables, should I also set
> > >> > kylin.route.hive.enabled to true to enable routing to Hive?
> > >> >
> > >> > Where can I find a list of all config vars? Looks like the
> > >> documentation is
> > >> > very sparse.
> > >> >
> > >> > Also, the webapp does not reflect kylin.properties well. Even after
> > >> > refreshes.... At least this was the case with 1.1incubating.
> > >> > On Aug 30, 2016 12:40 PM, "ShaoFeng Shi" <sh...@apache.org>
> > >> wrote:
> > >> >
> > >> > > Hi Sarnath, I totally agree with you; we're looking forward to see
> > >> your
> > >> > > sharing and patch.
> > >> > >
> > >> > > 2016-08-30 13:47 GMT+08:00 Sarnath K <st...@gmail.com>:
> > >> > >
> > >> > > > Sure. I just hope this never goes away. It is a great feature
> and
> > BI
> > >> > > tools
> > >> > > > like Tableau can work seamlessly. And, it is a point I keep
> > telling
> > >> > > > customers too. It is a great optimization that is really
> awesome.
> > >> > > > I will test it with Impala and post back.
> > >> > > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Best regards,
> > >> > >
> > >> > > Shaofeng Shi
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >> Shaofeng Shi
> > >>
> > >
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>

Re: Hive server vs Impala server

Posted by ShaoFeng Shi <sh...@apache.org>.
1. Could you please provide the error trace?

3. From a recent version Kylin start to support "select *" with the
aggregated data in cube.

2016-09-07 21:40 GMT+08:00 Sarnath K <st...@gmail.com>:

> Hi,
> Today I tested this routing feature. It did not work as expected.
>
> 1. When I execute Group By queries along with WHERE conditions on a plain
> cube  built without conditions,  the JDBC driver simply throws an
> exception.  I was expecting Kylin to redirect to impala which I had
> configured through hive.url configuration parameter. This did not happen. I
> see the code that tries olapenumerator and if that catches exception,
> redirects to Hive enumerator. But the output I see does not correspond to
> the code. What am I missing?
>
> 2. For the scenario described above,  I am able to catch the exception in
> Java app code and redirect to Impala. This works like a charm and is very
> fast. Not sure if self-service BI tools will do all these. Consider
> filtering a pivot chart.  This translates to a SQL with a WHERE followed by
> a GROUP BY. This is not a rare case but rather commonly occurring case.
>
> 3. When I issue 'select * from table',  Kylin returns cube data instead of
> giving raw table contents.  This is a bit unexpected.  But I remember
> reading somewhere in the forums earlier that this is actually a feature.
> Does my memory serve me right?
>
> Thanks,
> Best,
> Sarnath
> Sent from 1+3
>
> On Sep 2, 2016 9:37 AM, "Sarnath K" <st...@gmail.com> wrote:
>
> > Sure. I assume hive routing is enabled by default so that I should be
> able
> > to do 'select *' queries through JDBC. This is what we have been trying
> > with 1.1 and t did not work. The webapp does not use routing and it is
> > documented. No worries on the webapp part.
> >
> > Is there any docs on uninstalling Kylin -remove all HBase tables,
> > Meta-Data and wipe of all intermediate hive tables etc..?
> >
> > Also, I am using prebuilt binaries for HBase 0.98.
> > We are using HBase 0.98.6 that comes with CDH.. Hope this config works.
> > On Sep 2, 2016 6:43 AM, "ShaoFeng Shi" <sh...@apache.org> wrote:
> >
> >> Hi Sarnath,
> >>
> >> For kylin.sh on Ubuntu issue, there is an open JIRA (
> >> https://issues.apache.org/jira/browse/KYLIN-1978), would you like to
> >> contribute your finding as a patch? thank you!
> >>
> >> The "kylin.route.hive.enabled" isn't in use anymore; what need is just
> >> "hive.url", "hive.user" and "hive.password".
> >>
> >> All the config parameters can be found in the java
> >> class org.apache.kylin.common.KylinConfigBase.java; There is a work
> item
> >> about adding doc for these parameters, we will add it soon.
> >>
> >> Changing kylin.properties need restart kylin server to take effective;
> >> Besides I havn't heard such problem (even with 1.1), please retry and
> >> report problem with detail information and we can have a check.
> >>
> >>
> >> 2016-09-01 23:18 GMT+08:00 Sarnath K <st...@gmail.com>:
> >>
> >> > I just downloaded 1.5.3 version of Kylin. The startup scripts fail in
> >> > Ubuntu because of "sh" in place of "bash". Replaced that in 2 scripts
> to
> >> > get going.
> >> >
> >> > And, apart from hive.URL and other variables, should I also set
> >> > kylin.route.hive.enabled to true to enable routing to Hive?
> >> >
> >> > Where can I find a list of all config vars? Looks like the
> >> documentation is
> >> > very sparse.
> >> >
> >> > Also, the webapp does not reflect kylin.properties well. Even after
> >> > refreshes.... At least this was the case with 1.1incubating.
> >> > On Aug 30, 2016 12:40 PM, "ShaoFeng Shi" <sh...@apache.org>
> >> wrote:
> >> >
> >> > > Hi Sarnath, I totally agree with you; we're looking forward to see
> >> your
> >> > > sharing and patch.
> >> > >
> >> > > 2016-08-30 13:47 GMT+08:00 Sarnath K <st...@gmail.com>:
> >> > >
> >> > > > Sure. I just hope this never goes away. It is a great feature and
> BI
> >> > > tools
> >> > > > like Tableau can work seamlessly. And, it is a point I keep
> telling
> >> > > > customers too. It is a great optimization that is really awesome.
> >> > > > I will test it with Impala and post back.
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Best regards,
> >> > >
> >> > > Shaofeng Shi
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Best regards,
> >>
> >> Shaofeng Shi
> >>
> >
>



-- 
Best regards,

Shaofeng Shi

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
Hi,
Today I tested this routing feature. It did not work as expected.

1. When I execute Group By queries along with WHERE conditions on a plain
cube  built without conditions,  the JDBC driver simply throws an
exception.  I was expecting Kylin to redirect to impala which I had
configured through hive.url configuration parameter. This did not happen. I
see the code that tries olapenumerator and if that catches exception,
redirects to Hive enumerator. But the output I see does not correspond to
the code. What am I missing?

2. For the scenario described above,  I am able to catch the exception in
Java app code and redirect to Impala. This works like a charm and is very
fast. Not sure if self-service BI tools will do all these. Consider
filtering a pivot chart.  This translates to a SQL with a WHERE followed by
a GROUP BY. This is not a rare case but rather commonly occurring case.

3. When I issue 'select * from table',  Kylin returns cube data instead of
giving raw table contents.  This is a bit unexpected.  But I remember
reading somewhere in the forums earlier that this is actually a feature.
Does my memory serve me right?

Thanks,
Best,
Sarnath
Sent from 1+3

On Sep 2, 2016 9:37 AM, "Sarnath K" <st...@gmail.com> wrote:

> Sure. I assume hive routing is enabled by default so that I should be able
> to do 'select *' queries through JDBC. This is what we have been trying
> with 1.1 and t did not work. The webapp does not use routing and it is
> documented. No worries on the webapp part.
>
> Is there any docs on uninstalling Kylin -remove all HBase tables,
> Meta-Data and wipe of all intermediate hive tables etc..?
>
> Also, I am using prebuilt binaries for HBase 0.98.
> We are using HBase 0.98.6 that comes with CDH.. Hope this config works.
> On Sep 2, 2016 6:43 AM, "ShaoFeng Shi" <sh...@apache.org> wrote:
>
>> Hi Sarnath,
>>
>> For kylin.sh on Ubuntu issue, there is an open JIRA (
>> https://issues.apache.org/jira/browse/KYLIN-1978), would you like to
>> contribute your finding as a patch? thank you!
>>
>> The "kylin.route.hive.enabled" isn't in use anymore; what need is just
>> "hive.url", "hive.user" and "hive.password".
>>
>> All the config parameters can be found in the java
>> class org.apache.kylin.common.KylinConfigBase.java; There is a work item
>> about adding doc for these parameters, we will add it soon.
>>
>> Changing kylin.properties need restart kylin server to take effective;
>> Besides I havn't heard such problem (even with 1.1), please retry and
>> report problem with detail information and we can have a check.
>>
>>
>> 2016-09-01 23:18 GMT+08:00 Sarnath K <st...@gmail.com>:
>>
>> > I just downloaded 1.5.3 version of Kylin. The startup scripts fail in
>> > Ubuntu because of "sh" in place of "bash". Replaced that in 2 scripts to
>> > get going.
>> >
>> > And, apart from hive.URL and other variables, should I also set
>> > kylin.route.hive.enabled to true to enable routing to Hive?
>> >
>> > Where can I find a list of all config vars? Looks like the
>> documentation is
>> > very sparse.
>> >
>> > Also, the webapp does not reflect kylin.properties well. Even after
>> > refreshes.... At least this was the case with 1.1incubating.
>> > On Aug 30, 2016 12:40 PM, "ShaoFeng Shi" <sh...@apache.org>
>> wrote:
>> >
>> > > Hi Sarnath, I totally agree with you; we're looking forward to see
>> your
>> > > sharing and patch.
>> > >
>> > > 2016-08-30 13:47 GMT+08:00 Sarnath K <st...@gmail.com>:
>> > >
>> > > > Sure. I just hope this never goes away. It is a great feature and BI
>> > > tools
>> > > > like Tableau can work seamlessly. And, it is a point I keep telling
>> > > > customers too. It is a great optimization that is really awesome.
>> > > > I will test it with Impala and post back.
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > >
>> > > Shaofeng Shi
>> > >
>> >
>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi
>>
>

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
Sure. I assume hive routing is enabled by default so that I should be able
to do 'select *' queries through JDBC. This is what we have been trying
with 1.1 and t did not work. The webapp does not use routing and it is
documented. No worries on the webapp part.

Is there any docs on uninstalling Kylin -remove all HBase tables, Meta-Data
and wipe of all intermediate hive tables etc..?

Also, I am using prebuilt binaries for HBase 0.98.
We are using HBase 0.98.6 that comes with CDH.. Hope this config works.
On Sep 2, 2016 6:43 AM, "ShaoFeng Shi" <sh...@apache.org> wrote:

> Hi Sarnath,
>
> For kylin.sh on Ubuntu issue, there is an open JIRA (
> https://issues.apache.org/jira/browse/KYLIN-1978), would you like to
> contribute your finding as a patch? thank you!
>
> The "kylin.route.hive.enabled" isn't in use anymore; what need is just
> "hive.url", "hive.user" and "hive.password".
>
> All the config parameters can be found in the java
> class org.apache.kylin.common.KylinConfigBase.java; There is a work item
> about adding doc for these parameters, we will add it soon.
>
> Changing kylin.properties need restart kylin server to take effective;
> Besides I havn't heard such problem (even with 1.1), please retry and
> report problem with detail information and we can have a check.
>
>
> 2016-09-01 23:18 GMT+08:00 Sarnath K <st...@gmail.com>:
>
> > I just downloaded 1.5.3 version of Kylin. The startup scripts fail in
> > Ubuntu because of "sh" in place of "bash". Replaced that in 2 scripts to
> > get going.
> >
> > And, apart from hive.URL and other variables, should I also set
> > kylin.route.hive.enabled to true to enable routing to Hive?
> >
> > Where can I find a list of all config vars? Looks like the documentation
> is
> > very sparse.
> >
> > Also, the webapp does not reflect kylin.properties well. Even after
> > refreshes.... At least this was the case with 1.1incubating.
> > On Aug 30, 2016 12:40 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:
> >
> > > Hi Sarnath, I totally agree with you; we're looking forward to see your
> > > sharing and patch.
> > >
> > > 2016-08-30 13:47 GMT+08:00 Sarnath K <st...@gmail.com>:
> > >
> > > > Sure. I just hope this never goes away. It is a great feature and BI
> > > tools
> > > > like Tableau can work seamlessly. And, it is a point I keep telling
> > > > customers too. It is a great optimization that is really awesome.
> > > > I will test it with Impala and post back.
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > Shaofeng Shi
> > >
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>

Re: Hive server vs Impala server

Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Sarnath,

For kylin.sh on Ubuntu issue, there is an open JIRA (
https://issues.apache.org/jira/browse/KYLIN-1978), would you like to
contribute your finding as a patch? thank you!

The "kylin.route.hive.enabled" isn't in use anymore; what need is just
"hive.url", "hive.user" and "hive.password".

All the config parameters can be found in the java
class org.apache.kylin.common.KylinConfigBase.java; There is a work item
about adding doc for these parameters, we will add it soon.

Changing kylin.properties need restart kylin server to take effective;
Besides I havn't heard such problem (even with 1.1), please retry and
report problem with detail information and we can have a check.


2016-09-01 23:18 GMT+08:00 Sarnath K <st...@gmail.com>:

> I just downloaded 1.5.3 version of Kylin. The startup scripts fail in
> Ubuntu because of "sh" in place of "bash". Replaced that in 2 scripts to
> get going.
>
> And, apart from hive.URL and other variables, should I also set
> kylin.route.hive.enabled to true to enable routing to Hive?
>
> Where can I find a list of all config vars? Looks like the documentation is
> very sparse.
>
> Also, the webapp does not reflect kylin.properties well. Even after
> refreshes.... At least this was the case with 1.1incubating.
> On Aug 30, 2016 12:40 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:
>
> > Hi Sarnath, I totally agree with you; we're looking forward to see your
> > sharing and patch.
> >
> > 2016-08-30 13:47 GMT+08:00 Sarnath K <st...@gmail.com>:
> >
> > > Sure. I just hope this never goes away. It is a great feature and BI
> > tools
> > > like Tableau can work seamlessly. And, it is a point I keep telling
> > > customers too. It is a great optimization that is really awesome.
> > > I will test it with Impala and post back.
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> > Shaofeng Shi
> >
>



-- 
Best regards,

Shaofeng Shi

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
I just downloaded 1.5.3 version of Kylin. The startup scripts fail in
Ubuntu because of "sh" in place of "bash". Replaced that in 2 scripts to
get going.

And, apart from hive.URL and other variables, should I also set
kylin.route.hive.enabled to true to enable routing to Hive?

Where can I find a list of all config vars? Looks like the documentation is
very sparse.

Also, the webapp does not reflect kylin.properties well. Even after
refreshes.... At least this was the case with 1.1incubating.
On Aug 30, 2016 12:40 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:

> Hi Sarnath, I totally agree with you; we're looking forward to see your
> sharing and patch.
>
> 2016-08-30 13:47 GMT+08:00 Sarnath K <st...@gmail.com>:
>
> > Sure. I just hope this never goes away. It is a great feature and BI
> tools
> > like Tableau can work seamlessly. And, it is a point I keep telling
> > customers too. It is a great optimization that is really awesome.
> > I will test it with Impala and post back.
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>

Re: Hive server vs Impala server

Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Sarnath, I totally agree with you; we're looking forward to see your
sharing and patch.

2016-08-30 13:47 GMT+08:00 Sarnath K <st...@gmail.com>:

> Sure. I just hope this never goes away. It is a great feature and BI tools
> like Tableau can work seamlessly. And, it is a point I keep telling
> customers too. It is a great optimization that is really awesome.
> I will test it with Impala and post back.
>



-- 
Best regards,

Shaofeng Shi

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
Sure. I just hope this never goes away. It is a great feature and BI tools
like Tableau can work seamlessly. And, it is a point I keep telling
customers too. It is a great optimization that is really awesome.
I will test it with Impala and post back.

Re: Hive server vs Impala server

Posted by ShaoFeng Shi <sh...@apache.org>.
The main consideration is the user experience that it brings: some queries
are fast, some are very slow; We think it is better to immediately tell
user that cubes couldn't fulfill, so that user can aware and take action
(either modify the query or modify the cube), instead of waiting for a long
time.

The deprecation mean "not suggested", and we may change that some day (when
identify a better solution than Hive, like Impala or SparkSQL); The code is
still there, you can modify or extend it. Take a look
on org.apache.kylin.query.relnode.OLAPToEnumerableConverter.java

2016-08-30 11:56 GMT+08:00 Sarnath K <st...@gmail.com>:

> So, that means Kylin will throw an exception if the Cube cannot serve an
> SQL? How would you then seamlessly connect a BI tool like Tableau? Should
> those folks use try-catch and modify their software? I am appalled at this
> deprecation. Or may be, did I Understand the routing feature wrong? Routing
> to impala can make things lot better. Or one can configure Hive with Tez
> exec engine...
>
> On Tue, Aug 30, 2016, 07:24 ShaoFeng Shi <sh...@apache.org> wrote:
>
> > The "routing to Hive" feature is deprecated as its user experience is bad
> > (user doesn't aware it was routed, would think Kylin is hung).
> >
> > The alternative way is building such a router atop Kylin, which should be
> > simple like a try-catch block;
> >
> > 2016-08-29 23:54 GMT+08:00 Sarnath K <st...@gmail.com>:
> >
> > > It looks like this has to be tested through only a Java program using
> > JDBC
> > > driver. The SQL interface in Webapp, does not seem to support cube
> misses
> > > and backend hive handling.
> > > We will try again tomorrow.
> > >
> > > Today, I also verified the hive.URL parameter in the source in
> > > KylinConfigBase.java. but it has a note that the methods returning the
> > hive
> > > URL, user and passwd are deprecated. What is the alternative planned in
> > > their place?
> > > Thanks,
> > > Best,
> > > Sarnath
> > > On Aug 28, 2016 8:36 PM, "Sarnath K" <st...@gmail.com> wrote:
> > >
> > > > This is cool. I will have this tried this week. Thanks!
> > > > On Aug 28, 2016 7:36 PM, "ShaoFeng Shi" <sh...@apache.org>
> > wrote:
> > > >
> > > >> The current implementation only route to hive, via Hive JDBC (add
> > > >> hive.url,
> > > >> hive.user, hive.password in kylin.properties); but it should be
> doable
> > > to
> > > >> extend; Please open a JIRA to track this requirement, thanks!
> > > >>
> > > >> 2016-08-28 15:40 GMT+08:00 Sarnath K <st...@gmail.com>:
> > > >>
> > > >> > Yes, I am looking at routing cube-misses to impala. This can
> > > >> significantly
> > > >> > increase net performance. Impala can execute queries on Hive
> tables
> > as
> > > >> > well. Impala provides a JDBC driver as well.
> > > >> > If this is possible, I just want to know what config variables I
> > need
> > > to
> > > >> > set to make this happen.
> > > >> > Thanks for your time!
> > > >> >
> > > >> > Best,
> > > >> > Sarnath
> > > >> > On Aug 28, 2016 7:10 AM, "ShaoFeng Shi" <sh...@apache.org>
> > > wrote:
> > > >> >
> > > >> > > I guess Sarnath's data source is still in hive, he wants to
> route
> > > some
> > > >> > > queries (cube didn't catch) to Impala, instead of hive.
> @Sarnath,
> > > can
> > > >> you
> > > >> > > confirm? Thanks.
> > > >> > >
> > > >> > > 2016-08-28 7:20 GMT+08:00 Li Yang <li...@apache.org>:
> > > >> > >
> > > >> > > > Use Impala as input source requires code change. It's not just
> > > some
> > > >> > > > configuration. Kylin is designed to support multiple data
> > sources
> > > >> > however
> > > >> > > > implementation must be provided for each kind of data source.
> > > >> > > >
> > > >> > > > On Fri, Aug 26, 2016 at 8:35 PM, Sarnath K <
> stellium@gmail.com>
> > > >> wrote:
> > > >> > > >
> > > >> > > > > Thanks for your answer and time. I would be happy to give a
> > try.
> > > >> But
> > > >> > > what
> > > >> > > > > config value I need to change (the above one is okay?) And
> > from
> > > >> which
> > > >> > > > > version of Kylin is that config value supported? Sorry for
> > being
> > > >> > > ignorant
> > > >> > > > > here. Any help is appreciated. i have the infra with me.
> Just
> > > >> tell me
> > > >> > > > what
> > > >> > > > > config value I should set in kylin.properties?
> > > >> > > > > On Aug 26, 2016 1:44 PM, "ShaoFeng Shi" <
> > shaofengshi@apache.org
> > > >
> > > >> > > wrote:
> > > >> > > > >
> > > >> > > > > > Hi Sarnath,
> > > >> > > > > >
> > > >> > > > > > Routing to Impala looks better than routing to Hive; In
> > theory
> > > >> it
> > > >> > is
> > > >> > > > > doable
> > > >> > > > > > as Kylin use standard JDBC API, but we haven't tried that;
> > you
> > > >> can
> > > >> > > > have a
> > > >> > > > > > try and share your finding with the community, thanks!
> > > >> > > > > >
> > > >> > > > > > 2016-08-25 16:05 GMT+08:00 Sarnath K <stellium@gmail.com
> >:
> > > >> > > > > >
> > > >> > > > > > > Or in other words,
> > > >> > > > > > > Can I specify Impala JdBC server in
> kylin.Hive.jdbc.driver
> > > ???
> > > >> > > > > > > There is a comment saying 'not in use' in the Github
> > > >> > > kylin.properties
> > > >> > > > > > file
> > > >> > > > > > > under examples....
> > > >> > > > > > >
> > > >> > > > > > > Any inputs would be great!
> > > >> > > > > > >
> > > >> > > > > > > Best,
> > > >> > > > > > > Sarnath
> > > >> > > > > > > On Aug 25, 2016 11:30 AM, "Sarnath K" <
> stellium@gmail.com
> > >
> > > >> > wrote:
> > > >> > > > > > >
> > > >> > > > > > > > Hi,
> > > >> > > > > > > > I tried searching for this a lot but could find a
> > > convincing
> > > >> > > > answer.
> > > >> > > > > > > >
> > > >> > > > > > > > Can Kylin use Impala JDBC driver underneath to run
> > queries
> > > >> on
> > > >> > > Hive
> > > >> > > > > > table
> > > >> > > > > > > > instead of Hive server?
> > > >> > > > > > > >
> > > >> > > > > > > > If this is not possible, I would like to understand
> why
> > > >> this is
> > > >> > > not
> > > >> > > > > > > > possible and if there is a roadmap for this.
> > > >> > > > > > > >
> > > >> > > > > > > > Thanks,
> > > >> > > > > > > > Best,
> > > >> > > > > > > > Sarnath
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > --
> > > >> > > > > > Best regards,
> > > >> > > > > >
> > > >> > > > > > Shaofeng Shi
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > --
> > > >> > > Best regards,
> > > >> > >
> > > >> > > Shaofeng Shi
> > > >> > >
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Best regards,
> > > >>
> > > >> Shaofeng Shi
> > > >>
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> > Shaofeng Shi
> >
>



-- 
Best regards,

Shaofeng Shi

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
So, that means Kylin will throw an exception if the Cube cannot serve an
SQL? How would you then seamlessly connect a BI tool like Tableau? Should
those folks use try-catch and modify their software? I am appalled at this
deprecation. Or may be, did I Understand the routing feature wrong? Routing
to impala can make things lot better. Or one can configure Hive with Tez
exec engine...

On Tue, Aug 30, 2016, 07:24 ShaoFeng Shi <sh...@apache.org> wrote:

> The "routing to Hive" feature is deprecated as its user experience is bad
> (user doesn't aware it was routed, would think Kylin is hung).
>
> The alternative way is building such a router atop Kylin, which should be
> simple like a try-catch block;
>
> 2016-08-29 23:54 GMT+08:00 Sarnath K <st...@gmail.com>:
>
> > It looks like this has to be tested through only a Java program using
> JDBC
> > driver. The SQL interface in Webapp, does not seem to support cube misses
> > and backend hive handling.
> > We will try again tomorrow.
> >
> > Today, I also verified the hive.URL parameter in the source in
> > KylinConfigBase.java. but it has a note that the methods returning the
> hive
> > URL, user and passwd are deprecated. What is the alternative planned in
> > their place?
> > Thanks,
> > Best,
> > Sarnath
> > On Aug 28, 2016 8:36 PM, "Sarnath K" <st...@gmail.com> wrote:
> >
> > > This is cool. I will have this tried this week. Thanks!
> > > On Aug 28, 2016 7:36 PM, "ShaoFeng Shi" <sh...@apache.org>
> wrote:
> > >
> > >> The current implementation only route to hive, via Hive JDBC (add
> > >> hive.url,
> > >> hive.user, hive.password in kylin.properties); but it should be doable
> > to
> > >> extend; Please open a JIRA to track this requirement, thanks!
> > >>
> > >> 2016-08-28 15:40 GMT+08:00 Sarnath K <st...@gmail.com>:
> > >>
> > >> > Yes, I am looking at routing cube-misses to impala. This can
> > >> significantly
> > >> > increase net performance. Impala can execute queries on Hive tables
> as
> > >> > well. Impala provides a JDBC driver as well.
> > >> > If this is possible, I just want to know what config variables I
> need
> > to
> > >> > set to make this happen.
> > >> > Thanks for your time!
> > >> >
> > >> > Best,
> > >> > Sarnath
> > >> > On Aug 28, 2016 7:10 AM, "ShaoFeng Shi" <sh...@apache.org>
> > wrote:
> > >> >
> > >> > > I guess Sarnath's data source is still in hive, he wants to route
> > some
> > >> > > queries (cube didn't catch) to Impala, instead of hive. @Sarnath,
> > can
> > >> you
> > >> > > confirm? Thanks.
> > >> > >
> > >> > > 2016-08-28 7:20 GMT+08:00 Li Yang <li...@apache.org>:
> > >> > >
> > >> > > > Use Impala as input source requires code change. It's not just
> > some
> > >> > > > configuration. Kylin is designed to support multiple data
> sources
> > >> > however
> > >> > > > implementation must be provided for each kind of data source.
> > >> > > >
> > >> > > > On Fri, Aug 26, 2016 at 8:35 PM, Sarnath K <st...@gmail.com>
> > >> wrote:
> > >> > > >
> > >> > > > > Thanks for your answer and time. I would be happy to give a
> try.
> > >> But
> > >> > > what
> > >> > > > > config value I need to change (the above one is okay?) And
> from
> > >> which
> > >> > > > > version of Kylin is that config value supported? Sorry for
> being
> > >> > > ignorant
> > >> > > > > here. Any help is appreciated. i have the infra with me. Just
> > >> tell me
> > >> > > > what
> > >> > > > > config value I should set in kylin.properties?
> > >> > > > > On Aug 26, 2016 1:44 PM, "ShaoFeng Shi" <
> shaofengshi@apache.org
> > >
> > >> > > wrote:
> > >> > > > >
> > >> > > > > > Hi Sarnath,
> > >> > > > > >
> > >> > > > > > Routing to Impala looks better than routing to Hive; In
> theory
> > >> it
> > >> > is
> > >> > > > > doable
> > >> > > > > > as Kylin use standard JDBC API, but we haven't tried that;
> you
> > >> can
> > >> > > > have a
> > >> > > > > > try and share your finding with the community, thanks!
> > >> > > > > >
> > >> > > > > > 2016-08-25 16:05 GMT+08:00 Sarnath K <st...@gmail.com>:
> > >> > > > > >
> > >> > > > > > > Or in other words,
> > >> > > > > > > Can I specify Impala JdBC server in kylin.Hive.jdbc.driver
> > ???
> > >> > > > > > > There is a comment saying 'not in use' in the Github
> > >> > > kylin.properties
> > >> > > > > > file
> > >> > > > > > > under examples....
> > >> > > > > > >
> > >> > > > > > > Any inputs would be great!
> > >> > > > > > >
> > >> > > > > > > Best,
> > >> > > > > > > Sarnath
> > >> > > > > > > On Aug 25, 2016 11:30 AM, "Sarnath K" <stellium@gmail.com
> >
> > >> > wrote:
> > >> > > > > > >
> > >> > > > > > > > Hi,
> > >> > > > > > > > I tried searching for this a lot but could find a
> > convincing
> > >> > > > answer.
> > >> > > > > > > >
> > >> > > > > > > > Can Kylin use Impala JDBC driver underneath to run
> queries
> > >> on
> > >> > > Hive
> > >> > > > > > table
> > >> > > > > > > > instead of Hive server?
> > >> > > > > > > >
> > >> > > > > > > > If this is not possible, I would like to understand why
> > >> this is
> > >> > > not
> > >> > > > > > > > possible and if there is a roadmap for this.
> > >> > > > > > > >
> > >> > > > > > > > Thanks,
> > >> > > > > > > > Best,
> > >> > > > > > > > Sarnath
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > --
> > >> > > > > > Best regards,
> > >> > > > > >
> > >> > > > > > Shaofeng Shi
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Best regards,
> > >> > >
> > >> > > Shaofeng Shi
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >> Shaofeng Shi
> > >>
> > >
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>

Re: Hive server vs Impala server

Posted by ShaoFeng Shi <sh...@apache.org>.
The "routing to Hive" feature is deprecated as its user experience is bad
(user doesn't aware it was routed, would think Kylin is hung).

The alternative way is building such a router atop Kylin, which should be
simple like a try-catch block;

2016-08-29 23:54 GMT+08:00 Sarnath K <st...@gmail.com>:

> It looks like this has to be tested through only a Java program using JDBC
> driver. The SQL interface in Webapp, does not seem to support cube misses
> and backend hive handling.
> We will try again tomorrow.
>
> Today, I also verified the hive.URL parameter in the source in
> KylinConfigBase.java. but it has a note that the methods returning the hive
> URL, user and passwd are deprecated. What is the alternative planned in
> their place?
> Thanks,
> Best,
> Sarnath
> On Aug 28, 2016 8:36 PM, "Sarnath K" <st...@gmail.com> wrote:
>
> > This is cool. I will have this tried this week. Thanks!
> > On Aug 28, 2016 7:36 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:
> >
> >> The current implementation only route to hive, via Hive JDBC (add
> >> hive.url,
> >> hive.user, hive.password in kylin.properties); but it should be doable
> to
> >> extend; Please open a JIRA to track this requirement, thanks!
> >>
> >> 2016-08-28 15:40 GMT+08:00 Sarnath K <st...@gmail.com>:
> >>
> >> > Yes, I am looking at routing cube-misses to impala. This can
> >> significantly
> >> > increase net performance. Impala can execute queries on Hive tables as
> >> > well. Impala provides a JDBC driver as well.
> >> > If this is possible, I just want to know what config variables I need
> to
> >> > set to make this happen.
> >> > Thanks for your time!
> >> >
> >> > Best,
> >> > Sarnath
> >> > On Aug 28, 2016 7:10 AM, "ShaoFeng Shi" <sh...@apache.org>
> wrote:
> >> >
> >> > > I guess Sarnath's data source is still in hive, he wants to route
> some
> >> > > queries (cube didn't catch) to Impala, instead of hive. @Sarnath,
> can
> >> you
> >> > > confirm? Thanks.
> >> > >
> >> > > 2016-08-28 7:20 GMT+08:00 Li Yang <li...@apache.org>:
> >> > >
> >> > > > Use Impala as input source requires code change. It's not just
> some
> >> > > > configuration. Kylin is designed to support multiple data sources
> >> > however
> >> > > > implementation must be provided for each kind of data source.
> >> > > >
> >> > > > On Fri, Aug 26, 2016 at 8:35 PM, Sarnath K <st...@gmail.com>
> >> wrote:
> >> > > >
> >> > > > > Thanks for your answer and time. I would be happy to give a try.
> >> But
> >> > > what
> >> > > > > config value I need to change (the above one is okay?) And from
> >> which
> >> > > > > version of Kylin is that config value supported? Sorry for being
> >> > > ignorant
> >> > > > > here. Any help is appreciated. i have the infra with me. Just
> >> tell me
> >> > > > what
> >> > > > > config value I should set in kylin.properties?
> >> > > > > On Aug 26, 2016 1:44 PM, "ShaoFeng Shi" <shaofengshi@apache.org
> >
> >> > > wrote:
> >> > > > >
> >> > > > > > Hi Sarnath,
> >> > > > > >
> >> > > > > > Routing to Impala looks better than routing to Hive; In theory
> >> it
> >> > is
> >> > > > > doable
> >> > > > > > as Kylin use standard JDBC API, but we haven't tried that; you
> >> can
> >> > > > have a
> >> > > > > > try and share your finding with the community, thanks!
> >> > > > > >
> >> > > > > > 2016-08-25 16:05 GMT+08:00 Sarnath K <st...@gmail.com>:
> >> > > > > >
> >> > > > > > > Or in other words,
> >> > > > > > > Can I specify Impala JdBC server in kylin.Hive.jdbc.driver
> ???
> >> > > > > > > There is a comment saying 'not in use' in the Github
> >> > > kylin.properties
> >> > > > > > file
> >> > > > > > > under examples....
> >> > > > > > >
> >> > > > > > > Any inputs would be great!
> >> > > > > > >
> >> > > > > > > Best,
> >> > > > > > > Sarnath
> >> > > > > > > On Aug 25, 2016 11:30 AM, "Sarnath K" <st...@gmail.com>
> >> > wrote:
> >> > > > > > >
> >> > > > > > > > Hi,
> >> > > > > > > > I tried searching for this a lot but could find a
> convincing
> >> > > > answer.
> >> > > > > > > >
> >> > > > > > > > Can Kylin use Impala JDBC driver underneath to run queries
> >> on
> >> > > Hive
> >> > > > > > table
> >> > > > > > > > instead of Hive server?
> >> > > > > > > >
> >> > > > > > > > If this is not possible, I would like to understand why
> >> this is
> >> > > not
> >> > > > > > > > possible and if there is a roadmap for this.
> >> > > > > > > >
> >> > > > > > > > Thanks,
> >> > > > > > > > Best,
> >> > > > > > > > Sarnath
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > > Best regards,
> >> > > > > >
> >> > > > > > Shaofeng Shi
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Best regards,
> >> > >
> >> > > Shaofeng Shi
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Best regards,
> >>
> >> Shaofeng Shi
> >>
> >
>



-- 
Best regards,

Shaofeng Shi

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
It looks like this has to be tested through only a Java program using JDBC
driver. The SQL interface in Webapp, does not seem to support cube misses
and backend hive handling.
We will try again tomorrow.

Today, I also verified the hive.URL parameter in the source in
KylinConfigBase.java. but it has a note that the methods returning the hive
URL, user and passwd are deprecated. What is the alternative planned in
their place?
Thanks,
Best,
Sarnath
On Aug 28, 2016 8:36 PM, "Sarnath K" <st...@gmail.com> wrote:

> This is cool. I will have this tried this week. Thanks!
> On Aug 28, 2016 7:36 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:
>
>> The current implementation only route to hive, via Hive JDBC (add
>> hive.url,
>> hive.user, hive.password in kylin.properties); but it should be doable to
>> extend; Please open a JIRA to track this requirement, thanks!
>>
>> 2016-08-28 15:40 GMT+08:00 Sarnath K <st...@gmail.com>:
>>
>> > Yes, I am looking at routing cube-misses to impala. This can
>> significantly
>> > increase net performance. Impala can execute queries on Hive tables as
>> > well. Impala provides a JDBC driver as well.
>> > If this is possible, I just want to know what config variables I need to
>> > set to make this happen.
>> > Thanks for your time!
>> >
>> > Best,
>> > Sarnath
>> > On Aug 28, 2016 7:10 AM, "ShaoFeng Shi" <sh...@apache.org> wrote:
>> >
>> > > I guess Sarnath's data source is still in hive, he wants to route some
>> > > queries (cube didn't catch) to Impala, instead of hive. @Sarnath, can
>> you
>> > > confirm? Thanks.
>> > >
>> > > 2016-08-28 7:20 GMT+08:00 Li Yang <li...@apache.org>:
>> > >
>> > > > Use Impala as input source requires code change. It's not just some
>> > > > configuration. Kylin is designed to support multiple data sources
>> > however
>> > > > implementation must be provided for each kind of data source.
>> > > >
>> > > > On Fri, Aug 26, 2016 at 8:35 PM, Sarnath K <st...@gmail.com>
>> wrote:
>> > > >
>> > > > > Thanks for your answer and time. I would be happy to give a try.
>> But
>> > > what
>> > > > > config value I need to change (the above one is okay?) And from
>> which
>> > > > > version of Kylin is that config value supported? Sorry for being
>> > > ignorant
>> > > > > here. Any help is appreciated. i have the infra with me. Just
>> tell me
>> > > > what
>> > > > > config value I should set in kylin.properties?
>> > > > > On Aug 26, 2016 1:44 PM, "ShaoFeng Shi" <sh...@apache.org>
>> > > wrote:
>> > > > >
>> > > > > > Hi Sarnath,
>> > > > > >
>> > > > > > Routing to Impala looks better than routing to Hive; In theory
>> it
>> > is
>> > > > > doable
>> > > > > > as Kylin use standard JDBC API, but we haven't tried that; you
>> can
>> > > > have a
>> > > > > > try and share your finding with the community, thanks!
>> > > > > >
>> > > > > > 2016-08-25 16:05 GMT+08:00 Sarnath K <st...@gmail.com>:
>> > > > > >
>> > > > > > > Or in other words,
>> > > > > > > Can I specify Impala JdBC server in kylin.Hive.jdbc.driver ???
>> > > > > > > There is a comment saying 'not in use' in the Github
>> > > kylin.properties
>> > > > > > file
>> > > > > > > under examples....
>> > > > > > >
>> > > > > > > Any inputs would be great!
>> > > > > > >
>> > > > > > > Best,
>> > > > > > > Sarnath
>> > > > > > > On Aug 25, 2016 11:30 AM, "Sarnath K" <st...@gmail.com>
>> > wrote:
>> > > > > > >
>> > > > > > > > Hi,
>> > > > > > > > I tried searching for this a lot but could find a convincing
>> > > > answer.
>> > > > > > > >
>> > > > > > > > Can Kylin use Impala JDBC driver underneath to run queries
>> on
>> > > Hive
>> > > > > > table
>> > > > > > > > instead of Hive server?
>> > > > > > > >
>> > > > > > > > If this is not possible, I would like to understand why
>> this is
>> > > not
>> > > > > > > > possible and if there is a roadmap for this.
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > Best,
>> > > > > > > > Sarnath
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > Best regards,
>> > > > > >
>> > > > > > Shaofeng Shi
>> > > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > >
>> > > Shaofeng Shi
>> > >
>> >
>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi
>>
>

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
This is cool. I will have this tried this week. Thanks!
On Aug 28, 2016 7:36 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:

> The current implementation only route to hive, via Hive JDBC (add hive.url,
> hive.user, hive.password in kylin.properties); but it should be doable to
> extend; Please open a JIRA to track this requirement, thanks!
>
> 2016-08-28 15:40 GMT+08:00 Sarnath K <st...@gmail.com>:
>
> > Yes, I am looking at routing cube-misses to impala. This can
> significantly
> > increase net performance. Impala can execute queries on Hive tables as
> > well. Impala provides a JDBC driver as well.
> > If this is possible, I just want to know what config variables I need to
> > set to make this happen.
> > Thanks for your time!
> >
> > Best,
> > Sarnath
> > On Aug 28, 2016 7:10 AM, "ShaoFeng Shi" <sh...@apache.org> wrote:
> >
> > > I guess Sarnath's data source is still in hive, he wants to route some
> > > queries (cube didn't catch) to Impala, instead of hive. @Sarnath, can
> you
> > > confirm? Thanks.
> > >
> > > 2016-08-28 7:20 GMT+08:00 Li Yang <li...@apache.org>:
> > >
> > > > Use Impala as input source requires code change. It's not just some
> > > > configuration. Kylin is designed to support multiple data sources
> > however
> > > > implementation must be provided for each kind of data source.
> > > >
> > > > On Fri, Aug 26, 2016 at 8:35 PM, Sarnath K <st...@gmail.com>
> wrote:
> > > >
> > > > > Thanks for your answer and time. I would be happy to give a try.
> But
> > > what
> > > > > config value I need to change (the above one is okay?) And from
> which
> > > > > version of Kylin is that config value supported? Sorry for being
> > > ignorant
> > > > > here. Any help is appreciated. i have the infra with me. Just tell
> me
> > > > what
> > > > > config value I should set in kylin.properties?
> > > > > On Aug 26, 2016 1:44 PM, "ShaoFeng Shi" <sh...@apache.org>
> > > wrote:
> > > > >
> > > > > > Hi Sarnath,
> > > > > >
> > > > > > Routing to Impala looks better than routing to Hive; In theory it
> > is
> > > > > doable
> > > > > > as Kylin use standard JDBC API, but we haven't tried that; you
> can
> > > > have a
> > > > > > try and share your finding with the community, thanks!
> > > > > >
> > > > > > 2016-08-25 16:05 GMT+08:00 Sarnath K <st...@gmail.com>:
> > > > > >
> > > > > > > Or in other words,
> > > > > > > Can I specify Impala JdBC server in kylin.Hive.jdbc.driver ???
> > > > > > > There is a comment saying 'not in use' in the Github
> > > kylin.properties
> > > > > > file
> > > > > > > under examples....
> > > > > > >
> > > > > > > Any inputs would be great!
> > > > > > >
> > > > > > > Best,
> > > > > > > Sarnath
> > > > > > > On Aug 25, 2016 11:30 AM, "Sarnath K" <st...@gmail.com>
> > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > > I tried searching for this a lot but could find a convincing
> > > > answer.
> > > > > > > >
> > > > > > > > Can Kylin use Impala JDBC driver underneath to run queries on
> > > Hive
> > > > > > table
> > > > > > > > instead of Hive server?
> > > > > > > >
> > > > > > > > If this is not possible, I would like to understand why this
> is
> > > not
> > > > > > > > possible and if there is a roadmap for this.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Best,
> > > > > > > > Sarnath
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > >
> > > > > > Shaofeng Shi
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > Shaofeng Shi
> > >
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>

Re: Hive server vs Impala server

Posted by ShaoFeng Shi <sh...@apache.org>.
The current implementation only route to hive, via Hive JDBC (add hive.url,
hive.user, hive.password in kylin.properties); but it should be doable to
extend; Please open a JIRA to track this requirement, thanks!

2016-08-28 15:40 GMT+08:00 Sarnath K <st...@gmail.com>:

> Yes, I am looking at routing cube-misses to impala. This can significantly
> increase net performance. Impala can execute queries on Hive tables as
> well. Impala provides a JDBC driver as well.
> If this is possible, I just want to know what config variables I need to
> set to make this happen.
> Thanks for your time!
>
> Best,
> Sarnath
> On Aug 28, 2016 7:10 AM, "ShaoFeng Shi" <sh...@apache.org> wrote:
>
> > I guess Sarnath's data source is still in hive, he wants to route some
> > queries (cube didn't catch) to Impala, instead of hive. @Sarnath, can you
> > confirm? Thanks.
> >
> > 2016-08-28 7:20 GMT+08:00 Li Yang <li...@apache.org>:
> >
> > > Use Impala as input source requires code change. It's not just some
> > > configuration. Kylin is designed to support multiple data sources
> however
> > > implementation must be provided for each kind of data source.
> > >
> > > On Fri, Aug 26, 2016 at 8:35 PM, Sarnath K <st...@gmail.com> wrote:
> > >
> > > > Thanks for your answer and time. I would be happy to give a try. But
> > what
> > > > config value I need to change (the above one is okay?) And from which
> > > > version of Kylin is that config value supported? Sorry for being
> > ignorant
> > > > here. Any help is appreciated. i have the infra with me. Just tell me
> > > what
> > > > config value I should set in kylin.properties?
> > > > On Aug 26, 2016 1:44 PM, "ShaoFeng Shi" <sh...@apache.org>
> > wrote:
> > > >
> > > > > Hi Sarnath,
> > > > >
> > > > > Routing to Impala looks better than routing to Hive; In theory it
> is
> > > > doable
> > > > > as Kylin use standard JDBC API, but we haven't tried that; you can
> > > have a
> > > > > try and share your finding with the community, thanks!
> > > > >
> > > > > 2016-08-25 16:05 GMT+08:00 Sarnath K <st...@gmail.com>:
> > > > >
> > > > > > Or in other words,
> > > > > > Can I specify Impala JdBC server in kylin.Hive.jdbc.driver ???
> > > > > > There is a comment saying 'not in use' in the Github
> > kylin.properties
> > > > > file
> > > > > > under examples....
> > > > > >
> > > > > > Any inputs would be great!
> > > > > >
> > > > > > Best,
> > > > > > Sarnath
> > > > > > On Aug 25, 2016 11:30 AM, "Sarnath K" <st...@gmail.com>
> wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > > I tried searching for this a lot but could find a convincing
> > > answer.
> > > > > > >
> > > > > > > Can Kylin use Impala JDBC driver underneath to run queries on
> > Hive
> > > > > table
> > > > > > > instead of Hive server?
> > > > > > >
> > > > > > > If this is not possible, I would like to understand why this is
> > not
> > > > > > > possible and if there is a roadmap for this.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Best,
> > > > > > > Sarnath
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > > Shaofeng Shi
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> > Shaofeng Shi
> >
>



-- 
Best regards,

Shaofeng Shi

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
Yes, I am looking at routing cube-misses to impala. This can significantly
increase net performance. Impala can execute queries on Hive tables as
well. Impala provides a JDBC driver as well.
If this is possible, I just want to know what config variables I need to
set to make this happen.
Thanks for your time!

Best,
Sarnath
On Aug 28, 2016 7:10 AM, "ShaoFeng Shi" <sh...@apache.org> wrote:

> I guess Sarnath's data source is still in hive, he wants to route some
> queries (cube didn't catch) to Impala, instead of hive. @Sarnath, can you
> confirm? Thanks.
>
> 2016-08-28 7:20 GMT+08:00 Li Yang <li...@apache.org>:
>
> > Use Impala as input source requires code change. It's not just some
> > configuration. Kylin is designed to support multiple data sources however
> > implementation must be provided for each kind of data source.
> >
> > On Fri, Aug 26, 2016 at 8:35 PM, Sarnath K <st...@gmail.com> wrote:
> >
> > > Thanks for your answer and time. I would be happy to give a try. But
> what
> > > config value I need to change (the above one is okay?) And from which
> > > version of Kylin is that config value supported? Sorry for being
> ignorant
> > > here. Any help is appreciated. i have the infra with me. Just tell me
> > what
> > > config value I should set in kylin.properties?
> > > On Aug 26, 2016 1:44 PM, "ShaoFeng Shi" <sh...@apache.org>
> wrote:
> > >
> > > > Hi Sarnath,
> > > >
> > > > Routing to Impala looks better than routing to Hive; In theory it is
> > > doable
> > > > as Kylin use standard JDBC API, but we haven't tried that; you can
> > have a
> > > > try and share your finding with the community, thanks!
> > > >
> > > > 2016-08-25 16:05 GMT+08:00 Sarnath K <st...@gmail.com>:
> > > >
> > > > > Or in other words,
> > > > > Can I specify Impala JdBC server in kylin.Hive.jdbc.driver ???
> > > > > There is a comment saying 'not in use' in the Github
> kylin.properties
> > > > file
> > > > > under examples....
> > > > >
> > > > > Any inputs would be great!
> > > > >
> > > > > Best,
> > > > > Sarnath
> > > > > On Aug 25, 2016 11:30 AM, "Sarnath K" <st...@gmail.com> wrote:
> > > > >
> > > > > > Hi,
> > > > > > I tried searching for this a lot but could find a convincing
> > answer.
> > > > > >
> > > > > > Can Kylin use Impala JDBC driver underneath to run queries on
> Hive
> > > > table
> > > > > > instead of Hive server?
> > > > > >
> > > > > > If this is not possible, I would like to understand why this is
> not
> > > > > > possible and if there is a roadmap for this.
> > > > > >
> > > > > > Thanks,
> > > > > > Best,
> > > > > > Sarnath
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > > Shaofeng Shi
> > > >
> > >
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>

Re: Hive server vs Impala server

Posted by ShaoFeng Shi <sh...@apache.org>.
I guess Sarnath's data source is still in hive, he wants to route some
queries (cube didn't catch) to Impala, instead of hive. @Sarnath, can you
confirm? Thanks.

2016-08-28 7:20 GMT+08:00 Li Yang <li...@apache.org>:

> Use Impala as input source requires code change. It's not just some
> configuration. Kylin is designed to support multiple data sources however
> implementation must be provided for each kind of data source.
>
> On Fri, Aug 26, 2016 at 8:35 PM, Sarnath K <st...@gmail.com> wrote:
>
> > Thanks for your answer and time. I would be happy to give a try. But what
> > config value I need to change (the above one is okay?) And from which
> > version of Kylin is that config value supported? Sorry for being ignorant
> > here. Any help is appreciated. i have the infra with me. Just tell me
> what
> > config value I should set in kylin.properties?
> > On Aug 26, 2016 1:44 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:
> >
> > > Hi Sarnath,
> > >
> > > Routing to Impala looks better than routing to Hive; In theory it is
> > doable
> > > as Kylin use standard JDBC API, but we haven't tried that; you can
> have a
> > > try and share your finding with the community, thanks!
> > >
> > > 2016-08-25 16:05 GMT+08:00 Sarnath K <st...@gmail.com>:
> > >
> > > > Or in other words,
> > > > Can I specify Impala JdBC server in kylin.Hive.jdbc.driver ???
> > > > There is a comment saying 'not in use' in the Github kylin.properties
> > > file
> > > > under examples....
> > > >
> > > > Any inputs would be great!
> > > >
> > > > Best,
> > > > Sarnath
> > > > On Aug 25, 2016 11:30 AM, "Sarnath K" <st...@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > > I tried searching for this a lot but could find a convincing
> answer.
> > > > >
> > > > > Can Kylin use Impala JDBC driver underneath to run queries on Hive
> > > table
> > > > > instead of Hive server?
> > > > >
> > > > > If this is not possible, I would like to understand why this is not
> > > > > possible and if there is a roadmap for this.
> > > > >
> > > > > Thanks,
> > > > > Best,
> > > > > Sarnath
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > Shaofeng Shi
> > >
> >
>



-- 
Best regards,

Shaofeng Shi

Re: Hive server vs Impala server

Posted by Li Yang <li...@apache.org>.
Use Impala as input source requires code change. It's not just some
configuration. Kylin is designed to support multiple data sources however
implementation must be provided for each kind of data source.

On Fri, Aug 26, 2016 at 8:35 PM, Sarnath K <st...@gmail.com> wrote:

> Thanks for your answer and time. I would be happy to give a try. But what
> config value I need to change (the above one is okay?) And from which
> version of Kylin is that config value supported? Sorry for being ignorant
> here. Any help is appreciated. i have the infra with me. Just tell me what
> config value I should set in kylin.properties?
> On Aug 26, 2016 1:44 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:
>
> > Hi Sarnath,
> >
> > Routing to Impala looks better than routing to Hive; In theory it is
> doable
> > as Kylin use standard JDBC API, but we haven't tried that; you can have a
> > try and share your finding with the community, thanks!
> >
> > 2016-08-25 16:05 GMT+08:00 Sarnath K <st...@gmail.com>:
> >
> > > Or in other words,
> > > Can I specify Impala JdBC server in kylin.Hive.jdbc.driver ???
> > > There is a comment saying 'not in use' in the Github kylin.properties
> > file
> > > under examples....
> > >
> > > Any inputs would be great!
> > >
> > > Best,
> > > Sarnath
> > > On Aug 25, 2016 11:30 AM, "Sarnath K" <st...@gmail.com> wrote:
> > >
> > > > Hi,
> > > > I tried searching for this a lot but could find a convincing answer.
> > > >
> > > > Can Kylin use Impala JDBC driver underneath to run queries on Hive
> > table
> > > > instead of Hive server?
> > > >
> > > > If this is not possible, I would like to understand why this is not
> > > > possible and if there is a roadmap for this.
> > > >
> > > > Thanks,
> > > > Best,
> > > > Sarnath
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> > Shaofeng Shi
> >
>

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
Thanks for your answer and time. I would be happy to give a try. But what
config value I need to change (the above one is okay?) And from which
version of Kylin is that config value supported? Sorry for being ignorant
here. Any help is appreciated. i have the infra with me. Just tell me what
config value I should set in kylin.properties?
On Aug 26, 2016 1:44 PM, "ShaoFeng Shi" <sh...@apache.org> wrote:

> Hi Sarnath,
>
> Routing to Impala looks better than routing to Hive; In theory it is doable
> as Kylin use standard JDBC API, but we haven't tried that; you can have a
> try and share your finding with the community, thanks!
>
> 2016-08-25 16:05 GMT+08:00 Sarnath K <st...@gmail.com>:
>
> > Or in other words,
> > Can I specify Impala JdBC server in kylin.Hive.jdbc.driver ???
> > There is a comment saying 'not in use' in the Github kylin.properties
> file
> > under examples....
> >
> > Any inputs would be great!
> >
> > Best,
> > Sarnath
> > On Aug 25, 2016 11:30 AM, "Sarnath K" <st...@gmail.com> wrote:
> >
> > > Hi,
> > > I tried searching for this a lot but could find a convincing answer.
> > >
> > > Can Kylin use Impala JDBC driver underneath to run queries on Hive
> table
> > > instead of Hive server?
> > >
> > > If this is not possible, I would like to understand why this is not
> > > possible and if there is a roadmap for this.
> > >
> > > Thanks,
> > > Best,
> > > Sarnath
> > >
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>

Re: Hive server vs Impala server

Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Sarnath,

Routing to Impala looks better than routing to Hive; In theory it is doable
as Kylin use standard JDBC API, but we haven't tried that; you can have a
try and share your finding with the community, thanks!

2016-08-25 16:05 GMT+08:00 Sarnath K <st...@gmail.com>:

> Or in other words,
> Can I specify Impala JdBC server in kylin.Hive.jdbc.driver ???
> There is a comment saying 'not in use' in the Github kylin.properties file
> under examples....
>
> Any inputs would be great!
>
> Best,
> Sarnath
> On Aug 25, 2016 11:30 AM, "Sarnath K" <st...@gmail.com> wrote:
>
> > Hi,
> > I tried searching for this a lot but could find a convincing answer.
> >
> > Can Kylin use Impala JDBC driver underneath to run queries on Hive table
> > instead of Hive server?
> >
> > If this is not possible, I would like to understand why this is not
> > possible and if there is a roadmap for this.
> >
> > Thanks,
> > Best,
> > Sarnath
> >
>



-- 
Best regards,

Shaofeng Shi

Re: Hive server vs Impala server

Posted by Sarnath K <st...@gmail.com>.
Or in other words,
Can I specify Impala JdBC server in kylin.Hive.jdbc.driver ???
There is a comment saying 'not in use' in the Github kylin.properties file
under examples....

Any inputs would be great!

Best,
Sarnath
On Aug 25, 2016 11:30 AM, "Sarnath K" <st...@gmail.com> wrote:

> Hi,
> I tried searching for this a lot but could find a convincing answer.
>
> Can Kylin use Impala JDBC driver underneath to run queries on Hive table
> instead of Hive server?
>
> If this is not possible, I would like to understand why this is not
> possible and if there is a roadmap for this.
>
> Thanks,
> Best,
> Sarnath
>