You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@livy.apache.org by lk_hadoop <lk...@163.com> on 2017/11/22 13:42:21 UTC
livy can't list databases
hi,all:
I'm trying livy0.4 with spark2.1
curl -H "Content-type: application/json" -X POST http://kafka02:8998/sessions -d '{"kind": "spark"}' | python -m json.tool
curl -H "Content-type: application/json" -X POST http://kafka02:8998/sessions/0/statements -d '{"code": "spark.sql(\"show databases\").show"}' | python -m json.tool
curl http://kafka02:8998/sessions/0/statements/0 | python -m json.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 235 100 235 0 0 15959 0 --:--:-- --:--:-- --:--:-- 16785
{
"code": "spark.sql(\"show databases\").show",
"id": 0,
"output": {
"data": {
"text/plain": "+------------+\n|databaseName|\n+------------+\n| default|\n+------------+"
},
"execution_count": 0,
"status": "ok"
},
"progress": 1.0,
"state": "available"
}
It looks like can't read the metadata,I have config livy with SPARK_HOME,and run under yarn model,the hive-site.xml also cp to SPARK_HOME/conf/.
but't when I use spark-shell:
scala> spark.sql("show databases").show
+-------------+
| databaseName|
+-------------+
| default|
| tpcds_carbon|
|tpcds_carbon2|
| tpcds_indexr|
|tpcds_parquet|
| tpcds_source|
+-------------+
2017-11-22
lk_hadoop
Re: livy can't list databases
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Cool
thanks for the update.
Regards
JB
On 11/23/2017 01:52 AM, lk_hadoop wrote:
> Thank you jb@nanthrax.net <ma...@nanthrax.net> , I have resolved it by
> config livy.repl.enable-hive-context = true
> 2017-11-23
> --------------------------------------------------------------------------------
> lk_hadoop
> --------------------------------------------------------------------------------
>
> *发件人：*Jean-Baptiste Onofré <jb...@nanthrax.net>
> *发送时间：*2017-11-22 21:45
> *主题：*Re: livy can't list databases
> *收件人：*"user"<us...@livy.incubator.apache.org>
> *抄送：*
> Hi,
> let me take a look. Thanks for the report.
> Regards
> JB
> On 11/22/2017 02:42 PM, lk_hadoop wrote:
> > hi,all:
> > I'm trying livy0.4 with spark2.1
> > curl -H "Content-type: application/json" -X POST
> > http://kafka02:8998/sessions -d '{"kind": "spark"}' | python -m json.tool
> > curl -H "Content-type: application/json" -X POST
> > http://kafka02:8998/sessions/0/statements -d '{"code": "spark.sql(\"show
> > databases\").show"}' | python -m json.tool
> > curl http://kafka02:8998/sessions/0/statements/0 | python -m json.tool
> > % Total % Received % Xferd Average Speed Time Time Time Current
> > Dload Upload Total Spent Left Speed
> > 100 235 100 235 0 0 15959 0 --:--:-- --:--:-- --:--:-- 16785
> > {
> > "code": "spark.sql(\"show databases\").show",
> > "id": 0,
> > "output": {
> > "data": {
> > "text/plain": "+------------+\n|databaseName|\n+------------+\n|
> > default|\n+------------+"
> > },
> > "execution_count": 0,
> > "status": "ok"
> > },
> > "progress": 1.0,
> > "state": "available"
> > }
> > It looks like can't read the metadata,I have config livy with SPARK_HOME,and run
> > under yarn model,the hive-site.xml also cp to SPARK_HOME/conf/.
> > *but't when I use spark-shell:*
> > **
> > **
> > scala> spark.sql("show databases").show
> > +-------------+
> > | databaseName|
> > +-------------+
> > | default|
> > | tpcds_carbon|
> > |tpcds_carbon2|
> > | tpcds_indexr|
> > |tpcds_parquet|
> > | tpcds_source|
> > +-------------+
> > 2017-11-22
> > --------------------------------------------------------------------------------
> > lk_hadoop
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Re: livy can't list databases
Posted by lk_hadoop <lk...@163.com>.
Thank you jb@nanthrax.net , I have resolved it by config livy.repl.enable-hive-context = true
2017-11-23
lk_hadoop
发件人：Jean-Baptiste Onofré <jb...@nanthrax.net>
发送时间：2017-11-22 21:45
主题：Re: livy can't list databases
收件人："user"<us...@livy.incubator.apache.org>
抄送：
Hi,
let me take a look. Thanks for the report.
Regards
JB
On 11/22/2017 02:42 PM, lk_hadoop wrote:
> hi,all:
> I'm trying livy0.4 with spark2.1
> curl -H "Content-type: application/json" -X POST
> http://kafka02:8998/sessions -d '{"kind": "spark"}' | python -m json.tool
> curl -H "Content-type: application/json" -X POST
> http://kafka02:8998/sessions/0/statements -d '{"code": "spark.sql(\"show
> databases\").show"}' | python -m json.tool
> curl http://kafka02:8998/sessions/0/statements/0 | python -m json.tool
> % Total % Received % Xferd Average Speed Time Time Time Current
> Dload Upload Total Spent Left Speed
> 100 235 100 235 0 0 15959 0 --:--:-- --:--:-- --:--:-- 16785
> {
> "code": "spark.sql(\"show databases\").show",
> "id": 0,
> "output": {
> "data": {
> "text/plain": "+------------+\n|databaseName|\n+------------+\n|
> default|\n+------------+"
> },
> "execution_count": 0,
> "status": "ok"
> },
> "progress": 1.0,
> "state": "available"
> }
> It looks like can't read the metadata,I have config livy with SPARK_HOME,and run
> under yarn model,the hive-site.xml also cp to SPARK_HOME/conf/.
> *but't when I use spark-shell:*
> **
> **
> scala> spark.sql("show databases").show
> +-------------+
> | databaseName|
> +-------------+
> | default|
> | tpcds_carbon|
> |tpcds_carbon2|
> | tpcds_indexr|
> |tpcds_parquet|
> | tpcds_source|
> +-------------+
> 2017-11-22
> --------------------------------------------------------------------------------
> lk_hadoop
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: livy can't list databases
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi,
let me take a look. Thanks for the report.
Regards
JB
On 11/22/2017 02:42 PM, lk_hadoop wrote:
> hi,all:
> I'm trying livy0.4 with spark2.1
> curl -H "Content-type: application/json" -X POST
> http://kafka02:8998/sessions -d '{"kind": "spark"}' | python -m json.tool
> curl -H "Content-type: application/json" -X POST
> http://kafka02:8998/sessions/0/statements -d '{"code": "spark.sql(\"show
> databases\").show"}' | python -m json.tool
> curl http://kafka02:8998/sessions/0/statements/0 | python -m json.tool
> % Total % Received % Xferd Average Speed Time Time Time Current
> Dload Upload Total Spent Left Speed
> 100 235 100 235 0 0 15959 0 --:--:-- --:--:-- --:--:-- 16785
> {
> "code": "spark.sql(\"show databases\").show",
> "id": 0,
> "output": {
> "data": {
> "text/plain": "+------------+\n|databaseName|\n+------------+\n|
> default|\n+------------+"
> },
> "execution_count": 0,
> "status": "ok"
> },
> "progress": 1.0,
> "state": "available"
> }
> It looks like can't read the metadata,I have config livy with SPARK_HOME,and run
> under yarn model,the hive-site.xml also cp to SPARK_HOME/conf/.
> *but't when I use spark-shell:*
> **
> **
> scala> spark.sql("show databases").show
> +-------------+
> | databaseName|
> +-------------+
> | default|
> | tpcds_carbon|
> |tpcds_carbon2|
> | tpcds_indexr|
> |tpcds_parquet|
> | tpcds_source|
> +-------------+
> 2017-11-22
> --------------------------------------------------------------------------------
> lk_hadoop
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com