You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Quincy HSIEH (JIRA)" <ji...@apache.org> on 2017/07/26 12:06:03 UTC
[jira] [Comment Edited] (SPARK-9776) Another instance of Derby may
have already booted the database
[ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101597#comment-16101597 ]
Quincy HSIEH edited comment on SPARK-9776 at 7/26/17 12:05 PM:
---------------------------------------------------------------
Hi,
In case that someone who has the same issue and comes to this page : here is the workaround I use to resolve the problem :
1. Configure Spark with Yarn client mode.
2. Use FairScheduler in yarn-site.xml to enable multi-queques so that you can spark-submit job to different queue name with --queue <QUEUE_NAME_YOU_PREFER>
<property>
<name>yarn.resourcemanager.scheduler.class</name> <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
</property>
3. Switch derby to memory mode by modify the connection string in
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby*:memory*:<YOUR_STORAGE_DIR>/metastore_db;create=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
was (Author: quincy.tw):
Hi,
In case that someone who has the same issue and comes to this page : here is the workaround I use to resolve the problem :
1. Configure Spark with Yarn client mode.
2. Use FairScheduler in yarn-site.xml to enable multi-queques so that you can spark-submit job to different queue name with --queue <QUEUE_NAME_YOU_PREFER>
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<!--value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value-->
<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
</property>
3. Switch derby to memory mode by modify the connection string in
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby*:memory*:<YOUR_STORAGE_DIR>/metastore_db;create=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
> Another instance of Derby may have already booted the database
> ---------------------------------------------------------------
>
> Key: SPARK-9776
> URL: https://issues.apache.org/jira/browse/SPARK-9776
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.5.0
> Environment: Mac Yosemite, spark-1.5.0
> Reporter: Sudhakar Thota
> Attachments: SPARK-9776-FL1.rtf
>
>
> val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc) results in error. Though the same works for spark-1.4.1.
> Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org