running multiple hive jobs from same directory

[Vasilis Liaskovitis <vl...@gmail.com>]

Hi,

I am trying to run mutliple hive jobs in parallel from the same
working directory. A lot of the jobs fail with the following:

  FAILED: Error in metadata: javax.jdo.JDOFatalDataStoreException:
Failed to start database 'metastore_db',
  see the next exception for details.
  NestedThrowables:
  java.sql.SQLException: Failed to start database 'metastore_db', see
the next exception for details.
  FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
  Command exited with non-zero status 9

My guess is that  multiple hive processes are trying to create or
lock/write to the metastore_db directory. Is this needed? Is there a
way to run multiple jobs from the same working directory? any help is
welcome.
thanks,

- Vasilis

Re: running multiple hive jobs from same directory

[Edward Capriolo <ed...@gmail.com>]

On Thu, Jan 28, 2010 at 12:44 PM, Vasilis Liaskovitis
<vl...@gmail.com> wrote:
> Hi,
>
> I am trying to run mutliple hive jobs in parallel from the same
> working directory. A lot of the jobs fail with the following:
>
>  FAILED: Error in metadata: javax.jdo.JDOFatalDataStoreException:
> Failed to start database 'metastore_db',
>  see the next exception for details.
>  NestedThrowables:
>  java.sql.SQLException: Failed to start database 'metastore_db', see
> the next exception for details.
>  FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask
>  Command exited with non-zero status 9
>
> My guess is that  multiple hive processes are trying to create or
> lock/write to the metastore_db directory. Is this needed? Is there a
> way to run multiple jobs from the same working directory? any help is
> welcome.
> thanks,
>
> - Vasilis
>

It soulds like you are trying to use the embedded driver which is on by default.

To have multiple-concurrent access to the metastore refer to this.
http://wiki.apache.org/hadoop/HiveDerbyServerMode