You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Gautam Singaraju <ga...@gmail.com> on 2012/07/18 21:39:32 UTC

Oozie Mysql connection

Hi Oozie community:


I am working on getting Oozie working with MySQL; and I have followed
the instructions in a previous post:
http://mail-archives.apache.org/mod_mbox/incubator-oozie-users/201110.mbox/%3CCAC8XGpEFCTEnHZjdbPQoKgWm-jiK0gzY=A_9WdJGZUzPjEfnAA@mail.gmail.com%3E
(I have copied some part of the suggestions at the end of the email.)

I am using Oozie 3.2 and mysql 5.1.

Exceptions:
1. I have installed oozie as linux user: hadoop.
2. also started it with webconsole.
3. I am running on AWS.

When I start oozie, and submit a job, I am getting the following error:

WARN AuthenticationFilter:381 - Authentication exception:
AuthenticationToken expired
org.apache.hadoop.security.authentication.client.AuthenticationException:
AuthenticationToken expired

Any suggestions would be greatly appreciated. Additionally, Apache
Oozie can benefit from a writeup on how to get oozie and mysql
working.

Thanks in advance!
<--------------snip>

1. Get a jdbc connector jar from here:
http://dev.mysql.com/downloads/connector/j/
2. Run the following command (as user oozie):
          sudo  -u oozie <path-to-oozie-installation/oozie-setup.sh
-jars <path-to-connector-jar>
3. Create the following DB in your MySQL and grant privs on it:
            # mysql -u root -p
           create database oozie;
           grant all privileges on oozie.* to 'oozie'@'localhost'
identified by 'oozie';
4. Replace the following properties in your oozie-site.xml with the
values give below:

    <property>
        <name>oozie.service.StoreService.jdbc.driver</name>
        <value>com.mysql.jdbc.Driver</value>
        <description>
            JDBC driver class.
        </description>
    </property>

    <property>
        <name>oozie.service.StoreService.jdbc.url</name>
        <value>jdbc:mysql://localhost:3306/oozie</value>
        <description>
            JDBC URL.
        </description>
    </property>

    <property>
        <name>oozie.service.StoreService.jdbc.username</name>
        <value>oozie</value>
        <description>
            DB user name.
        </description>
    </property>

    <property>
        <name>oozie.service.StoreService.jdbc.password</name>
        <value>oozie</value>
        <description>
            DB user password.

            IMPORTANT: if password is emtpy leave a 1 space string, the service
trims the value,
                       if empty Configuration assumes it is NULL.

            IMPORTANT: if the StoreServicePasswordService is active,
it will reset
this value with the value given in
                       the console.
        </description>
    </property>

<-----------------end snip>


---
Gautam

Re: Oozie Mysql connection

Posted by Gautam Singaraju <ga...@gmail.com>.
Hi Mohammed,

Thank you, that worked.

Regards,
---
Gautam


On Thu, Jul 26, 2012 at 1:07 AM, Mohammad Islam <mi...@yahoo.com> wrote:
> Hi Gautam,
> StoreService is no longer used in Oozie. It is being replace by JPAService. Your JPAService is configured to use derby.
> Please remove those and replace "StoreService" with "JPAService" and then restart Oozie.
>
> Regards
> Mohammad
>
> ----- Original Message -----
> From: Gautam Singaraju <ga...@gmail.com>
> To: oozie-users@incubator.apache.org
> Cc:
> Sent: Wednesday, July 25, 2012 6:06 PM
> Subject: Re: Oozie Mysql connection
>
> Hi Alejandro,
>
> Thanks for your suggestion. Hive (mysql metastore) is working with Oozie.
> I have the oozie-site.xml without any derby configuration; but mysql
> JPA settings. When I start oozie, it creates an derby instance in
> data/oozie-db. I log into the oozie's mysql server, and there is no
> tables there.
>
>
> From Settings from the oozie web console:
>
> Name: oozie.service.JPAService.jdbc.driver
> Value: org.apache.derby.jdbc.EmbeddedDriver
>
> Name: oozie.service.JPAService.jdbc.url
> Value: jdbc:derby:${oozie.data.dir}/${oozie.db.schema.name}-db;create=true
>
>
> Name: oozie.service.StoreService.jdbc.url
> Value: jdbc:mysql://<IP>:3306/oozie
>
> Name: oozie.service.StoreService.jdbc.driver
> Value: com.mysql.jdbc.Driver
>
> Any suggestions would be appreciated.
> ---
> Gautam
>
>
> On Thu, Jul 19, 2012 at 8:29 PM, Alejandro Abdelnur <tu...@cloudera.com> wrote:
>> Gautam,
>>
>> Look at the oozie hive launcher map log, that should give you information
>> about what exactly Hive is using.
>>
>> thx
>>
>> On Thu, Jul 19, 2012 at 6:04 PM, Gautam Singaraju <
>> gautam.singaraju@gmail.com> wrote:
>>
>>> Oops, that's a stupid question. Thanks, Alejandro.
>>> I am still puzzled how my oozie-MySQL config processed a job when I
>>> removed the derby conf and only had MySQL config; there were no tables
>>> created in MySQL...
>>> ---
>>> Gautam
>>>
>>>
>>> On Thu, Jul 19, 2012 at 5:47 PM, Alejandro Abdelnur <tu...@cloudera.com>
>>> wrote:
>>> > Gautam,
>>> >
>>> > You are confusing the Oozie server configuration in oozie-site.xml (which
>>> > uses mysql) with the hive action metatstore configuration which by
>>> default
>>> > uses derby. Those are 2 different databases.
>>> >
>>> > hope this helps
>>> >
>>> > thx
>>> >
>>> > On Thu, Jul 19, 2012 at 4:03 PM, Gautam Singaraju <
>>> > gautam.singaraju@gmail.com> wrote:
>>> >
>>> >> Thank you Harsh, for your help. I was able to redo some steps and was
>>> >> able to start oozie with the MySQL configuration as described.
>>> >> However, something very weird seems to be happening. Once I schedule a
>>> >> job, the job completes successfully. But, when I log into MySQL client
>>> >> to see the schemas, I dont see anything. Then I noticed that I had the
>>> >> following in the configuration:
>>> >>
>>> >> <property>
>>> >>   <name>hive.stats.dbclass</name>
>>> >>   <value>jdbc:derby</value>
>>> >>   <description>The default database that stores temporary hive
>>> >> statistics.</description>
>>> >> </property>
>>> >>
>>> >> <property>
>>> >>   <name>hive.stats.autogather</name>
>>> >>   <value>true</value>
>>> >>   <description>A flag to gather statistics automatically during the
>>> >> INSERT OVERWRITE command.</description>
>>> >> </property>
>>> >>
>>> >> <property>
>>> >>   <name>hive.stats.jdbcdriver</name>
>>> >>   <value>org.apache.derby.jdbc.EmbeddedDriver</value>
>>> >>   <description>The JDBC driver for the database that stores temporary
>>> >> hive statistics.</description>
>>> >> </property>
>>> >>
>>> >> <property>
>>> >>   <name>hive.stats.dbconnectionstring</name>
>>> >>   <value>jdbc:derby:;databaseName=TempStatsStore;create=true</value>
>>> >>   <description>The default connection string for the database that
>>> >> stores temporary hive statistics.</description>
>>> >> </property>
>>> >>
>>> >>
>>> >> Could someone explain what is happening here? Does oozie store temp
>>> >> jobs in a derby db and flush to mysql at a predetermined interval?
>>> >>
>>> >> Thank you,
>>> >> ---
>>> >> Gautam
>>> >>
>>> >>
>>> >> On Thu, Jul 19, 2012 at 3:45 AM, Harsh J <ha...@cloudera.com> wrote:
>>> >> > Hey Gautam,
>>> >> >
>>> >> > Can you also attach a server side oozie.log snippet after running an
>>> >> Oozie
>>> >> > job that seems to fail with the error you've sent?
>>> >> >
>>> >> > I do not think your issue is related to MySQL at this point, but
>>> server
>>> >> > logs will help determine. Thanks!
>>> >> >
>>> >> > On Thu, Jul 19, 2012 at 1:09 AM, Gautam Singaraju <
>>> >> > gautam.singaraju@gmail.com> wrote:
>>> >> >
>>> >> >> Hi Oozie community:
>>> >> >>
>>> >> >>
>>> >> >> I am working on getting Oozie working with MySQL; and I have followed
>>> >> >> the instructions in a previous post:
>>> >> >>
>>> >> >>
>>> >>
>>> http://mail-archives.apache.org/mod_mbox/incubator-oozie-users/201110.mbox/%3CCAC8XGpEFCTEnHZjdbPQoKgWm-jiK0gzY=A_9WdJGZUzPjEfnAA@mail.gmail.com%3E
>>> >> >> (I have copied some part of the suggestions at the end of the email.)
>>> >> >>
>>> >> >> I am using Oozie 3.2 and mysql 5.1.
>>> >> >>
>>> >> >> Exceptions:
>>> >> >> 1. I have installed oozie as linux user: hadoop.
>>> >> >> 2. also started it with webconsole.
>>> >> >> 3. I am running on AWS.
>>> >> >>
>>> >> >> When I start oozie, and submit a job, I am getting the following
>>> error:
>>> >> >>
>>> >> >> WARN AuthenticationFilter:381 - Authentication exception:
>>> >> >> AuthenticationToken expired
>>> >> >>
>>> >>
>>> org.apache.hadoop.security.authentication.client.AuthenticationException:
>>> >> >> AuthenticationToken expired
>>> >> >>
>>> >> >> Any suggestions would be greatly appreciated. Additionally, Apache
>>> >> >> Oozie can benefit from a writeup on how to get oozie and mysql
>>> >> >> working.
>>> >> >>
>>> >> >> Thanks in advance!
>>> >> >> <--------------snip>
>>> >> >>
>>> >> >> 1. Get a jdbc connector jar from here:
>>> >> >> http://dev.mysql.com/downloads/connector/j/
>>> >> >> 2. Run the following command (as user oozie):
>>> >> >>           sudo  -u oozie <path-to-oozie-installation/oozie-setup.sh
>>> >> >> -jars <path-to-connector-jar>
>>> >> >> 3. Create the following DB in your MySQL and grant privs on it:
>>> >> >>             # mysql -u root -p
>>> >> >>            create database oozie;
>>> >> >>            grant all privileges on oozie.* to 'oozie'@'localhost'
>>> >> >> identified by 'oozie';
>>> >> >> 4. Replace the following properties in your oozie-site.xml with the
>>> >> >> values give below:
>>> >> >>
>>> >> >>     <property>
>>> >> >>         <name>oozie.service.StoreService.jdbc.driver</name>
>>> >> >>         <value>com.mysql.jdbc.Driver</value>
>>> >> >>         <description>
>>> >> >>             JDBC driver class.
>>> >> >>         </description>
>>> >> >>     </property>
>>> >> >>
>>> >> >>     <property>
>>> >> >>         <name>oozie.service.StoreService.jdbc.url</name>
>>> >> >>         <value>jdbc:mysql://localhost:3306/oozie</value>
>>> >> >>         <description>
>>> >> >>             JDBC URL.
>>> >> >>         </description>
>>> >> >>     </property>
>>> >> >>
>>> >> >>     <property>
>>> >> >>         <name>oozie.service.StoreService.jdbc.username</name>
>>> >> >>         <value>oozie</value>
>>> >> >>         <description>
>>> >> >>             DB user name.
>>> >> >>         </description>
>>> >> >>     </property>
>>> >> >>
>>> >> >>     <property>
>>> >> >>         <name>oozie.service.StoreService.jdbc.password</name>
>>> >> >>         <value>oozie</value>
>>> >> >>         <description>
>>> >> >>             DB user password.
>>> >> >>
>>> >> >>             IMPORTANT: if password is emtpy leave a 1 space string,
>>> the
>>> >> >> service
>>> >> >> trims the value,
>>> >> >>                        if empty Configuration assumes it is NULL.
>>> >> >>
>>> >> >>             IMPORTANT: if the StoreServicePasswordService is active,
>>> >> >> it will reset
>>> >> >> this value with the value given in
>>> >> >>                        the console.
>>> >> >>         </description>
>>> >> >>     </property>
>>> >> >>
>>> >> >> <-----------------end snip>
>>> >> >>
>>> >> >>
>>> >> >> ---
>>> >> >> Gautam
>>> >> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Harsh J
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Alejandro
>>>
>>
>>
>>
>> --
>> Alejandro
>

Re: Oozie Mysql connection

Posted by Mohammad Islam <mi...@yahoo.com>.
Hi Gautam,
StoreService is no longer used in Oozie. It is being replace by JPAService. Your JPAService is configured to use derby.
Please remove those and replace "StoreService" with "JPAService" and then restart Oozie.

Regards
Mohammad

----- Original Message -----
From: Gautam Singaraju <ga...@gmail.com>
To: oozie-users@incubator.apache.org
Cc: 
Sent: Wednesday, July 25, 2012 6:06 PM
Subject: Re: Oozie Mysql connection

Hi Alejandro,

Thanks for your suggestion. Hive (mysql metastore) is working with Oozie.
I have the oozie-site.xml without any derby configuration; but mysql
JPA settings. When I start oozie, it creates an derby instance in
data/oozie-db. I log into the oozie's mysql server, and there is no
tables there.


From Settings from the oozie web console:

Name: oozie.service.JPAService.jdbc.driver
Value: org.apache.derby.jdbc.EmbeddedDriver

Name: oozie.service.JPAService.jdbc.url
Value: jdbc:derby:${oozie.data.dir}/${oozie.db.schema.name}-db;create=true


Name: oozie.service.StoreService.jdbc.url
Value: jdbc:mysql://<IP>:3306/oozie

Name: oozie.service.StoreService.jdbc.driver
Value: com.mysql.jdbc.Driver

Any suggestions would be appreciated.
---
Gautam


On Thu, Jul 19, 2012 at 8:29 PM, Alejandro Abdelnur <tu...@cloudera.com> wrote:
> Gautam,
>
> Look at the oozie hive launcher map log, that should give you information
> about what exactly Hive is using.
>
> thx
>
> On Thu, Jul 19, 2012 at 6:04 PM, Gautam Singaraju <
> gautam.singaraju@gmail.com> wrote:
>
>> Oops, that's a stupid question. Thanks, Alejandro.
>> I am still puzzled how my oozie-MySQL config processed a job when I
>> removed the derby conf and only had MySQL config; there were no tables
>> created in MySQL...
>> ---
>> Gautam
>>
>>
>> On Thu, Jul 19, 2012 at 5:47 PM, Alejandro Abdelnur <tu...@cloudera.com>
>> wrote:
>> > Gautam,
>> >
>> > You are confusing the Oozie server configuration in oozie-site.xml (which
>> > uses mysql) with the hive action metatstore configuration which by
>> default
>> > uses derby. Those are 2 different databases.
>> >
>> > hope this helps
>> >
>> > thx
>> >
>> > On Thu, Jul 19, 2012 at 4:03 PM, Gautam Singaraju <
>> > gautam.singaraju@gmail.com> wrote:
>> >
>> >> Thank you Harsh, for your help. I was able to redo some steps and was
>> >> able to start oozie with the MySQL configuration as described.
>> >> However, something very weird seems to be happening. Once I schedule a
>> >> job, the job completes successfully. But, when I log into MySQL client
>> >> to see the schemas, I dont see anything. Then I noticed that I had the
>> >> following in the configuration:
>> >>
>> >> <property>
>> >>   <name>hive.stats.dbclass</name>
>> >>   <value>jdbc:derby</value>
>> >>   <description>The default database that stores temporary hive
>> >> statistics.</description>
>> >> </property>
>> >>
>> >> <property>
>> >>   <name>hive.stats.autogather</name>
>> >>   <value>true</value>
>> >>   <description>A flag to gather statistics automatically during the
>> >> INSERT OVERWRITE command.</description>
>> >> </property>
>> >>
>> >> <property>
>> >>   <name>hive.stats.jdbcdriver</name>
>> >>   <value>org.apache.derby.jdbc.EmbeddedDriver</value>
>> >>   <description>The JDBC driver for the database that stores temporary
>> >> hive statistics.</description>
>> >> </property>
>> >>
>> >> <property>
>> >>   <name>hive.stats.dbconnectionstring</name>
>> >>   <value>jdbc:derby:;databaseName=TempStatsStore;create=true</value>
>> >>   <description>The default connection string for the database that
>> >> stores temporary hive statistics.</description>
>> >> </property>
>> >>
>> >>
>> >> Could someone explain what is happening here? Does oozie store temp
>> >> jobs in a derby db and flush to mysql at a predetermined interval?
>> >>
>> >> Thank you,
>> >> ---
>> >> Gautam
>> >>
>> >>
>> >> On Thu, Jul 19, 2012 at 3:45 AM, Harsh J <ha...@cloudera.com> wrote:
>> >> > Hey Gautam,
>> >> >
>> >> > Can you also attach a server side oozie.log snippet after running an
>> >> Oozie
>> >> > job that seems to fail with the error you've sent?
>> >> >
>> >> > I do not think your issue is related to MySQL at this point, but
>> server
>> >> > logs will help determine. Thanks!
>> >> >
>> >> > On Thu, Jul 19, 2012 at 1:09 AM, Gautam Singaraju <
>> >> > gautam.singaraju@gmail.com> wrote:
>> >> >
>> >> >> Hi Oozie community:
>> >> >>
>> >> >>
>> >> >> I am working on getting Oozie working with MySQL; and I have followed
>> >> >> the instructions in a previous post:
>> >> >>
>> >> >>
>> >>
>> http://mail-archives.apache.org/mod_mbox/incubator-oozie-users/201110.mbox/%3CCAC8XGpEFCTEnHZjdbPQoKgWm-jiK0gzY=A_9WdJGZUzPjEfnAA@mail.gmail.com%3E
>> >> >> (I have copied some part of the suggestions at the end of the email.)
>> >> >>
>> >> >> I am using Oozie 3.2 and mysql 5.1.
>> >> >>
>> >> >> Exceptions:
>> >> >> 1. I have installed oozie as linux user: hadoop.
>> >> >> 2. also started it with webconsole.
>> >> >> 3. I am running on AWS.
>> >> >>
>> >> >> When I start oozie, and submit a job, I am getting the following
>> error:
>> >> >>
>> >> >> WARN AuthenticationFilter:381 - Authentication exception:
>> >> >> AuthenticationToken expired
>> >> >>
>> >>
>> org.apache.hadoop.security.authentication.client.AuthenticationException:
>> >> >> AuthenticationToken expired
>> >> >>
>> >> >> Any suggestions would be greatly appreciated. Additionally, Apache
>> >> >> Oozie can benefit from a writeup on how to get oozie and mysql
>> >> >> working.
>> >> >>
>> >> >> Thanks in advance!
>> >> >> <--------------snip>
>> >> >>
>> >> >> 1. Get a jdbc connector jar from here:
>> >> >> http://dev.mysql.com/downloads/connector/j/
>> >> >> 2. Run the following command (as user oozie):
>> >> >>           sudo  -u oozie <path-to-oozie-installation/oozie-setup.sh
>> >> >> -jars <path-to-connector-jar>
>> >> >> 3. Create the following DB in your MySQL and grant privs on it:
>> >> >>             # mysql -u root -p
>> >> >>            create database oozie;
>> >> >>            grant all privileges on oozie.* to 'oozie'@'localhost'
>> >> >> identified by 'oozie';
>> >> >> 4. Replace the following properties in your oozie-site.xml with the
>> >> >> values give below:
>> >> >>
>> >> >>     <property>
>> >> >>         <name>oozie.service.StoreService.jdbc.driver</name>
>> >> >>         <value>com.mysql.jdbc.Driver</value>
>> >> >>         <description>
>> >> >>             JDBC driver class.
>> >> >>         </description>
>> >> >>     </property>
>> >> >>
>> >> >>     <property>
>> >> >>         <name>oozie.service.StoreService.jdbc.url</name>
>> >> >>         <value>jdbc:mysql://localhost:3306/oozie</value>
>> >> >>         <description>
>> >> >>             JDBC URL.
>> >> >>         </description>
>> >> >>     </property>
>> >> >>
>> >> >>     <property>
>> >> >>         <name>oozie.service.StoreService.jdbc.username</name>
>> >> >>         <value>oozie</value>
>> >> >>         <description>
>> >> >>             DB user name.
>> >> >>         </description>
>> >> >>     </property>
>> >> >>
>> >> >>     <property>
>> >> >>         <name>oozie.service.StoreService.jdbc.password</name>
>> >> >>         <value>oozie</value>
>> >> >>         <description>
>> >> >>             DB user password.
>> >> >>
>> >> >>             IMPORTANT: if password is emtpy leave a 1 space string,
>> the
>> >> >> service
>> >> >> trims the value,
>> >> >>                        if empty Configuration assumes it is NULL.
>> >> >>
>> >> >>             IMPORTANT: if the StoreServicePasswordService is active,
>> >> >> it will reset
>> >> >> this value with the value given in
>> >> >>                        the console.
>> >> >>         </description>
>> >> >>     </property>
>> >> >>
>> >> >> <-----------------end snip>
>> >> >>
>> >> >>
>> >> >> ---
>> >> >> Gautam
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Harsh J
>> >>
>> >
>> >
>> >
>> > --
>> > Alejandro
>>
>
>
>
> --
> Alejandro


Re: Oozie Mysql connection

Posted by Gautam Singaraju <ga...@gmail.com>.
Hi Alejandro,

Thanks for your suggestion. Hive (mysql metastore) is working with Oozie.
I have the oozie-site.xml without any derby configuration; but mysql
JPA settings. When I start oozie, it creates an derby instance in
data/oozie-db. I log into the oozie's mysql server, and there is no
tables there.


>From Settings from the oozie web console:

Name: oozie.service.JPAService.jdbc.driver
Value: org.apache.derby.jdbc.EmbeddedDriver

Name: oozie.service.JPAService.jdbc.url
Value: jdbc:derby:${oozie.data.dir}/${oozie.db.schema.name}-db;create=true


Name: oozie.service.StoreService.jdbc.url
Value: jdbc:mysql://<IP>:3306/oozie

Name: oozie.service.StoreService.jdbc.driver
Value: com.mysql.jdbc.Driver

Any suggestions would be appreciated.
---
Gautam


On Thu, Jul 19, 2012 at 8:29 PM, Alejandro Abdelnur <tu...@cloudera.com> wrote:
> Gautam,
>
> Look at the oozie hive launcher map log, that should give you information
> about what exactly Hive is using.
>
> thx
>
> On Thu, Jul 19, 2012 at 6:04 PM, Gautam Singaraju <
> gautam.singaraju@gmail.com> wrote:
>
>> Oops, that's a stupid question. Thanks, Alejandro.
>> I am still puzzled how my oozie-MySQL config processed a job when I
>> removed the derby conf and only had MySQL config; there were no tables
>> created in MySQL...
>> ---
>> Gautam
>>
>>
>> On Thu, Jul 19, 2012 at 5:47 PM, Alejandro Abdelnur <tu...@cloudera.com>
>> wrote:
>> > Gautam,
>> >
>> > You are confusing the Oozie server configuration in oozie-site.xml (which
>> > uses mysql) with the hive action metatstore configuration which by
>> default
>> > uses derby. Those are 2 different databases.
>> >
>> > hope this helps
>> >
>> > thx
>> >
>> > On Thu, Jul 19, 2012 at 4:03 PM, Gautam Singaraju <
>> > gautam.singaraju@gmail.com> wrote:
>> >
>> >> Thank you Harsh, for your help. I was able to redo some steps and was
>> >> able to start oozie with the MySQL configuration as described.
>> >> However, something very weird seems to be happening. Once I schedule a
>> >> job, the job completes successfully. But, when I log into MySQL client
>> >> to see the schemas, I dont see anything. Then I noticed that I had the
>> >> following in the configuration:
>> >>
>> >> <property>
>> >>   <name>hive.stats.dbclass</name>
>> >>   <value>jdbc:derby</value>
>> >>   <description>The default database that stores temporary hive
>> >> statistics.</description>
>> >> </property>
>> >>
>> >> <property>
>> >>   <name>hive.stats.autogather</name>
>> >>   <value>true</value>
>> >>   <description>A flag to gather statistics automatically during the
>> >> INSERT OVERWRITE command.</description>
>> >> </property>
>> >>
>> >> <property>
>> >>   <name>hive.stats.jdbcdriver</name>
>> >>   <value>org.apache.derby.jdbc.EmbeddedDriver</value>
>> >>   <description>The JDBC driver for the database that stores temporary
>> >> hive statistics.</description>
>> >> </property>
>> >>
>> >> <property>
>> >>   <name>hive.stats.dbconnectionstring</name>
>> >>   <value>jdbc:derby:;databaseName=TempStatsStore;create=true</value>
>> >>   <description>The default connection string for the database that
>> >> stores temporary hive statistics.</description>
>> >> </property>
>> >>
>> >>
>> >> Could someone explain what is happening here? Does oozie store temp
>> >> jobs in a derby db and flush to mysql at a predetermined interval?
>> >>
>> >> Thank you,
>> >> ---
>> >> Gautam
>> >>
>> >>
>> >> On Thu, Jul 19, 2012 at 3:45 AM, Harsh J <ha...@cloudera.com> wrote:
>> >> > Hey Gautam,
>> >> >
>> >> > Can you also attach a server side oozie.log snippet after running an
>> >> Oozie
>> >> > job that seems to fail with the error you've sent?
>> >> >
>> >> > I do not think your issue is related to MySQL at this point, but
>> server
>> >> > logs will help determine. Thanks!
>> >> >
>> >> > On Thu, Jul 19, 2012 at 1:09 AM, Gautam Singaraju <
>> >> > gautam.singaraju@gmail.com> wrote:
>> >> >
>> >> >> Hi Oozie community:
>> >> >>
>> >> >>
>> >> >> I am working on getting Oozie working with MySQL; and I have followed
>> >> >> the instructions in a previous post:
>> >> >>
>> >> >>
>> >>
>> http://mail-archives.apache.org/mod_mbox/incubator-oozie-users/201110.mbox/%3CCAC8XGpEFCTEnHZjdbPQoKgWm-jiK0gzY=A_9WdJGZUzPjEfnAA@mail.gmail.com%3E
>> >> >> (I have copied some part of the suggestions at the end of the email.)
>> >> >>
>> >> >> I am using Oozie 3.2 and mysql 5.1.
>> >> >>
>> >> >> Exceptions:
>> >> >> 1. I have installed oozie as linux user: hadoop.
>> >> >> 2. also started it with webconsole.
>> >> >> 3. I am running on AWS.
>> >> >>
>> >> >> When I start oozie, and submit a job, I am getting the following
>> error:
>> >> >>
>> >> >> WARN AuthenticationFilter:381 - Authentication exception:
>> >> >> AuthenticationToken expired
>> >> >>
>> >>
>> org.apache.hadoop.security.authentication.client.AuthenticationException:
>> >> >> AuthenticationToken expired
>> >> >>
>> >> >> Any suggestions would be greatly appreciated. Additionally, Apache
>> >> >> Oozie can benefit from a writeup on how to get oozie and mysql
>> >> >> working.
>> >> >>
>> >> >> Thanks in advance!
>> >> >> <--------------snip>
>> >> >>
>> >> >> 1. Get a jdbc connector jar from here:
>> >> >> http://dev.mysql.com/downloads/connector/j/
>> >> >> 2. Run the following command (as user oozie):
>> >> >>           sudo  -u oozie <path-to-oozie-installation/oozie-setup.sh
>> >> >> -jars <path-to-connector-jar>
>> >> >> 3. Create the following DB in your MySQL and grant privs on it:
>> >> >>             # mysql -u root -p
>> >> >>            create database oozie;
>> >> >>            grant all privileges on oozie.* to 'oozie'@'localhost'
>> >> >> identified by 'oozie';
>> >> >> 4. Replace the following properties in your oozie-site.xml with the
>> >> >> values give below:
>> >> >>
>> >> >>     <property>
>> >> >>         <name>oozie.service.StoreService.jdbc.driver</name>
>> >> >>         <value>com.mysql.jdbc.Driver</value>
>> >> >>         <description>
>> >> >>             JDBC driver class.
>> >> >>         </description>
>> >> >>     </property>
>> >> >>
>> >> >>     <property>
>> >> >>         <name>oozie.service.StoreService.jdbc.url</name>
>> >> >>         <value>jdbc:mysql://localhost:3306/oozie</value>
>> >> >>         <description>
>> >> >>             JDBC URL.
>> >> >>         </description>
>> >> >>     </property>
>> >> >>
>> >> >>     <property>
>> >> >>         <name>oozie.service.StoreService.jdbc.username</name>
>> >> >>         <value>oozie</value>
>> >> >>         <description>
>> >> >>             DB user name.
>> >> >>         </description>
>> >> >>     </property>
>> >> >>
>> >> >>     <property>
>> >> >>         <name>oozie.service.StoreService.jdbc.password</name>
>> >> >>         <value>oozie</value>
>> >> >>         <description>
>> >> >>             DB user password.
>> >> >>
>> >> >>             IMPORTANT: if password is emtpy leave a 1 space string,
>> the
>> >> >> service
>> >> >> trims the value,
>> >> >>                        if empty Configuration assumes it is NULL.
>> >> >>
>> >> >>             IMPORTANT: if the StoreServicePasswordService is active,
>> >> >> it will reset
>> >> >> this value with the value given in
>> >> >>                        the console.
>> >> >>         </description>
>> >> >>     </property>
>> >> >>
>> >> >> <-----------------end snip>
>> >> >>
>> >> >>
>> >> >> ---
>> >> >> Gautam
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Harsh J
>> >>
>> >
>> >
>> >
>> > --
>> > Alejandro
>>
>
>
>
> --
> Alejandro

Re: Oozie Mysql connection

Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Gautam,

Look at the oozie hive launcher map log, that should give you information
about what exactly Hive is using.

thx

On Thu, Jul 19, 2012 at 6:04 PM, Gautam Singaraju <
gautam.singaraju@gmail.com> wrote:

> Oops, that's a stupid question. Thanks, Alejandro.
> I am still puzzled how my oozie-MySQL config processed a job when I
> removed the derby conf and only had MySQL config; there were no tables
> created in MySQL...
> ---
> Gautam
>
>
> On Thu, Jul 19, 2012 at 5:47 PM, Alejandro Abdelnur <tu...@cloudera.com>
> wrote:
> > Gautam,
> >
> > You are confusing the Oozie server configuration in oozie-site.xml (which
> > uses mysql) with the hive action metatstore configuration which by
> default
> > uses derby. Those are 2 different databases.
> >
> > hope this helps
> >
> > thx
> >
> > On Thu, Jul 19, 2012 at 4:03 PM, Gautam Singaraju <
> > gautam.singaraju@gmail.com> wrote:
> >
> >> Thank you Harsh, for your help. I was able to redo some steps and was
> >> able to start oozie with the MySQL configuration as described.
> >> However, something very weird seems to be happening. Once I schedule a
> >> job, the job completes successfully. But, when I log into MySQL client
> >> to see the schemas, I dont see anything. Then I noticed that I had the
> >> following in the configuration:
> >>
> >> <property>
> >>   <name>hive.stats.dbclass</name>
> >>   <value>jdbc:derby</value>
> >>   <description>The default database that stores temporary hive
> >> statistics.</description>
> >> </property>
> >>
> >> <property>
> >>   <name>hive.stats.autogather</name>
> >>   <value>true</value>
> >>   <description>A flag to gather statistics automatically during the
> >> INSERT OVERWRITE command.</description>
> >> </property>
> >>
> >> <property>
> >>   <name>hive.stats.jdbcdriver</name>
> >>   <value>org.apache.derby.jdbc.EmbeddedDriver</value>
> >>   <description>The JDBC driver for the database that stores temporary
> >> hive statistics.</description>
> >> </property>
> >>
> >> <property>
> >>   <name>hive.stats.dbconnectionstring</name>
> >>   <value>jdbc:derby:;databaseName=TempStatsStore;create=true</value>
> >>   <description>The default connection string for the database that
> >> stores temporary hive statistics.</description>
> >> </property>
> >>
> >>
> >> Could someone explain what is happening here? Does oozie store temp
> >> jobs in a derby db and flush to mysql at a predetermined interval?
> >>
> >> Thank you,
> >> ---
> >> Gautam
> >>
> >>
> >> On Thu, Jul 19, 2012 at 3:45 AM, Harsh J <ha...@cloudera.com> wrote:
> >> > Hey Gautam,
> >> >
> >> > Can you also attach a server side oozie.log snippet after running an
> >> Oozie
> >> > job that seems to fail with the error you've sent?
> >> >
> >> > I do not think your issue is related to MySQL at this point, but
> server
> >> > logs will help determine. Thanks!
> >> >
> >> > On Thu, Jul 19, 2012 at 1:09 AM, Gautam Singaraju <
> >> > gautam.singaraju@gmail.com> wrote:
> >> >
> >> >> Hi Oozie community:
> >> >>
> >> >>
> >> >> I am working on getting Oozie working with MySQL; and I have followed
> >> >> the instructions in a previous post:
> >> >>
> >> >>
> >>
> http://mail-archives.apache.org/mod_mbox/incubator-oozie-users/201110.mbox/%3CCAC8XGpEFCTEnHZjdbPQoKgWm-jiK0gzY=A_9WdJGZUzPjEfnAA@mail.gmail.com%3E
> >> >> (I have copied some part of the suggestions at the end of the email.)
> >> >>
> >> >> I am using Oozie 3.2 and mysql 5.1.
> >> >>
> >> >> Exceptions:
> >> >> 1. I have installed oozie as linux user: hadoop.
> >> >> 2. also started it with webconsole.
> >> >> 3. I am running on AWS.
> >> >>
> >> >> When I start oozie, and submit a job, I am getting the following
> error:
> >> >>
> >> >> WARN AuthenticationFilter:381 - Authentication exception:
> >> >> AuthenticationToken expired
> >> >>
> >>
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> >> >> AuthenticationToken expired
> >> >>
> >> >> Any suggestions would be greatly appreciated. Additionally, Apache
> >> >> Oozie can benefit from a writeup on how to get oozie and mysql
> >> >> working.
> >> >>
> >> >> Thanks in advance!
> >> >> <--------------snip>
> >> >>
> >> >> 1. Get a jdbc connector jar from here:
> >> >> http://dev.mysql.com/downloads/connector/j/
> >> >> 2. Run the following command (as user oozie):
> >> >>           sudo  -u oozie <path-to-oozie-installation/oozie-setup.sh
> >> >> -jars <path-to-connector-jar>
> >> >> 3. Create the following DB in your MySQL and grant privs on it:
> >> >>             # mysql -u root -p
> >> >>            create database oozie;
> >> >>            grant all privileges on oozie.* to 'oozie'@'localhost'
> >> >> identified by 'oozie';
> >> >> 4. Replace the following properties in your oozie-site.xml with the
> >> >> values give below:
> >> >>
> >> >>     <property>
> >> >>         <name>oozie.service.StoreService.jdbc.driver</name>
> >> >>         <value>com.mysql.jdbc.Driver</value>
> >> >>         <description>
> >> >>             JDBC driver class.
> >> >>         </description>
> >> >>     </property>
> >> >>
> >> >>     <property>
> >> >>         <name>oozie.service.StoreService.jdbc.url</name>
> >> >>         <value>jdbc:mysql://localhost:3306/oozie</value>
> >> >>         <description>
> >> >>             JDBC URL.
> >> >>         </description>
> >> >>     </property>
> >> >>
> >> >>     <property>
> >> >>         <name>oozie.service.StoreService.jdbc.username</name>
> >> >>         <value>oozie</value>
> >> >>         <description>
> >> >>             DB user name.
> >> >>         </description>
> >> >>     </property>
> >> >>
> >> >>     <property>
> >> >>         <name>oozie.service.StoreService.jdbc.password</name>
> >> >>         <value>oozie</value>
> >> >>         <description>
> >> >>             DB user password.
> >> >>
> >> >>             IMPORTANT: if password is emtpy leave a 1 space string,
> the
> >> >> service
> >> >> trims the value,
> >> >>                        if empty Configuration assumes it is NULL.
> >> >>
> >> >>             IMPORTANT: if the StoreServicePasswordService is active,
> >> >> it will reset
> >> >> this value with the value given in
> >> >>                        the console.
> >> >>         </description>
> >> >>     </property>
> >> >>
> >> >> <-----------------end snip>
> >> >>
> >> >>
> >> >> ---
> >> >> Gautam
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Harsh J
> >>
> >
> >
> >
> > --
> > Alejandro
>



-- 
Alejandro

Re: Oozie Mysql connection

Posted by Gautam Singaraju <ga...@gmail.com>.
Oops, that's a stupid question. Thanks, Alejandro.
I am still puzzled how my oozie-MySQL config processed a job when I
removed the derby conf and only had MySQL config; there were no tables
created in MySQL...
---
Gautam


On Thu, Jul 19, 2012 at 5:47 PM, Alejandro Abdelnur <tu...@cloudera.com> wrote:
> Gautam,
>
> You are confusing the Oozie server configuration in oozie-site.xml (which
> uses mysql) with the hive action metatstore configuration which by default
> uses derby. Those are 2 different databases.
>
> hope this helps
>
> thx
>
> On Thu, Jul 19, 2012 at 4:03 PM, Gautam Singaraju <
> gautam.singaraju@gmail.com> wrote:
>
>> Thank you Harsh, for your help. I was able to redo some steps and was
>> able to start oozie with the MySQL configuration as described.
>> However, something very weird seems to be happening. Once I schedule a
>> job, the job completes successfully. But, when I log into MySQL client
>> to see the schemas, I dont see anything. Then I noticed that I had the
>> following in the configuration:
>>
>> <property>
>>   <name>hive.stats.dbclass</name>
>>   <value>jdbc:derby</value>
>>   <description>The default database that stores temporary hive
>> statistics.</description>
>> </property>
>>
>> <property>
>>   <name>hive.stats.autogather</name>
>>   <value>true</value>
>>   <description>A flag to gather statistics automatically during the
>> INSERT OVERWRITE command.</description>
>> </property>
>>
>> <property>
>>   <name>hive.stats.jdbcdriver</name>
>>   <value>org.apache.derby.jdbc.EmbeddedDriver</value>
>>   <description>The JDBC driver for the database that stores temporary
>> hive statistics.</description>
>> </property>
>>
>> <property>
>>   <name>hive.stats.dbconnectionstring</name>
>>   <value>jdbc:derby:;databaseName=TempStatsStore;create=true</value>
>>   <description>The default connection string for the database that
>> stores temporary hive statistics.</description>
>> </property>
>>
>>
>> Could someone explain what is happening here? Does oozie store temp
>> jobs in a derby db and flush to mysql at a predetermined interval?
>>
>> Thank you,
>> ---
>> Gautam
>>
>>
>> On Thu, Jul 19, 2012 at 3:45 AM, Harsh J <ha...@cloudera.com> wrote:
>> > Hey Gautam,
>> >
>> > Can you also attach a server side oozie.log snippet after running an
>> Oozie
>> > job that seems to fail with the error you've sent?
>> >
>> > I do not think your issue is related to MySQL at this point, but server
>> > logs will help determine. Thanks!
>> >
>> > On Thu, Jul 19, 2012 at 1:09 AM, Gautam Singaraju <
>> > gautam.singaraju@gmail.com> wrote:
>> >
>> >> Hi Oozie community:
>> >>
>> >>
>> >> I am working on getting Oozie working with MySQL; and I have followed
>> >> the instructions in a previous post:
>> >>
>> >>
>> http://mail-archives.apache.org/mod_mbox/incubator-oozie-users/201110.mbox/%3CCAC8XGpEFCTEnHZjdbPQoKgWm-jiK0gzY=A_9WdJGZUzPjEfnAA@mail.gmail.com%3E
>> >> (I have copied some part of the suggestions at the end of the email.)
>> >>
>> >> I am using Oozie 3.2 and mysql 5.1.
>> >>
>> >> Exceptions:
>> >> 1. I have installed oozie as linux user: hadoop.
>> >> 2. also started it with webconsole.
>> >> 3. I am running on AWS.
>> >>
>> >> When I start oozie, and submit a job, I am getting the following error:
>> >>
>> >> WARN AuthenticationFilter:381 - Authentication exception:
>> >> AuthenticationToken expired
>> >>
>> org.apache.hadoop.security.authentication.client.AuthenticationException:
>> >> AuthenticationToken expired
>> >>
>> >> Any suggestions would be greatly appreciated. Additionally, Apache
>> >> Oozie can benefit from a writeup on how to get oozie and mysql
>> >> working.
>> >>
>> >> Thanks in advance!
>> >> <--------------snip>
>> >>
>> >> 1. Get a jdbc connector jar from here:
>> >> http://dev.mysql.com/downloads/connector/j/
>> >> 2. Run the following command (as user oozie):
>> >>           sudo  -u oozie <path-to-oozie-installation/oozie-setup.sh
>> >> -jars <path-to-connector-jar>
>> >> 3. Create the following DB in your MySQL and grant privs on it:
>> >>             # mysql -u root -p
>> >>            create database oozie;
>> >>            grant all privileges on oozie.* to 'oozie'@'localhost'
>> >> identified by 'oozie';
>> >> 4. Replace the following properties in your oozie-site.xml with the
>> >> values give below:
>> >>
>> >>     <property>
>> >>         <name>oozie.service.StoreService.jdbc.driver</name>
>> >>         <value>com.mysql.jdbc.Driver</value>
>> >>         <description>
>> >>             JDBC driver class.
>> >>         </description>
>> >>     </property>
>> >>
>> >>     <property>
>> >>         <name>oozie.service.StoreService.jdbc.url</name>
>> >>         <value>jdbc:mysql://localhost:3306/oozie</value>
>> >>         <description>
>> >>             JDBC URL.
>> >>         </description>
>> >>     </property>
>> >>
>> >>     <property>
>> >>         <name>oozie.service.StoreService.jdbc.username</name>
>> >>         <value>oozie</value>
>> >>         <description>
>> >>             DB user name.
>> >>         </description>
>> >>     </property>
>> >>
>> >>     <property>
>> >>         <name>oozie.service.StoreService.jdbc.password</name>
>> >>         <value>oozie</value>
>> >>         <description>
>> >>             DB user password.
>> >>
>> >>             IMPORTANT: if password is emtpy leave a 1 space string, the
>> >> service
>> >> trims the value,
>> >>                        if empty Configuration assumes it is NULL.
>> >>
>> >>             IMPORTANT: if the StoreServicePasswordService is active,
>> >> it will reset
>> >> this value with the value given in
>> >>                        the console.
>> >>         </description>
>> >>     </property>
>> >>
>> >> <-----------------end snip>
>> >>
>> >>
>> >> ---
>> >> Gautam
>> >>
>> >
>> >
>> >
>> > --
>> > Harsh J
>>
>
>
>
> --
> Alejandro

Re: Oozie Mysql connection

Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Gautam,

You are confusing the Oozie server configuration in oozie-site.xml (which
uses mysql) with the hive action metatstore configuration which by default
uses derby. Those are 2 different databases.

hope this helps

thx

On Thu, Jul 19, 2012 at 4:03 PM, Gautam Singaraju <
gautam.singaraju@gmail.com> wrote:

> Thank you Harsh, for your help. I was able to redo some steps and was
> able to start oozie with the MySQL configuration as described.
> However, something very weird seems to be happening. Once I schedule a
> job, the job completes successfully. But, when I log into MySQL client
> to see the schemas, I dont see anything. Then I noticed that I had the
> following in the configuration:
>
> <property>
>   <name>hive.stats.dbclass</name>
>   <value>jdbc:derby</value>
>   <description>The default database that stores temporary hive
> statistics.</description>
> </property>
>
> <property>
>   <name>hive.stats.autogather</name>
>   <value>true</value>
>   <description>A flag to gather statistics automatically during the
> INSERT OVERWRITE command.</description>
> </property>
>
> <property>
>   <name>hive.stats.jdbcdriver</name>
>   <value>org.apache.derby.jdbc.EmbeddedDriver</value>
>   <description>The JDBC driver for the database that stores temporary
> hive statistics.</description>
> </property>
>
> <property>
>   <name>hive.stats.dbconnectionstring</name>
>   <value>jdbc:derby:;databaseName=TempStatsStore;create=true</value>
>   <description>The default connection string for the database that
> stores temporary hive statistics.</description>
> </property>
>
>
> Could someone explain what is happening here? Does oozie store temp
> jobs in a derby db and flush to mysql at a predetermined interval?
>
> Thank you,
> ---
> Gautam
>
>
> On Thu, Jul 19, 2012 at 3:45 AM, Harsh J <ha...@cloudera.com> wrote:
> > Hey Gautam,
> >
> > Can you also attach a server side oozie.log snippet after running an
> Oozie
> > job that seems to fail with the error you've sent?
> >
> > I do not think your issue is related to MySQL at this point, but server
> > logs will help determine. Thanks!
> >
> > On Thu, Jul 19, 2012 at 1:09 AM, Gautam Singaraju <
> > gautam.singaraju@gmail.com> wrote:
> >
> >> Hi Oozie community:
> >>
> >>
> >> I am working on getting Oozie working with MySQL; and I have followed
> >> the instructions in a previous post:
> >>
> >>
> http://mail-archives.apache.org/mod_mbox/incubator-oozie-users/201110.mbox/%3CCAC8XGpEFCTEnHZjdbPQoKgWm-jiK0gzY=A_9WdJGZUzPjEfnAA@mail.gmail.com%3E
> >> (I have copied some part of the suggestions at the end of the email.)
> >>
> >> I am using Oozie 3.2 and mysql 5.1.
> >>
> >> Exceptions:
> >> 1. I have installed oozie as linux user: hadoop.
> >> 2. also started it with webconsole.
> >> 3. I am running on AWS.
> >>
> >> When I start oozie, and submit a job, I am getting the following error:
> >>
> >> WARN AuthenticationFilter:381 - Authentication exception:
> >> AuthenticationToken expired
> >>
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> >> AuthenticationToken expired
> >>
> >> Any suggestions would be greatly appreciated. Additionally, Apache
> >> Oozie can benefit from a writeup on how to get oozie and mysql
> >> working.
> >>
> >> Thanks in advance!
> >> <--------------snip>
> >>
> >> 1. Get a jdbc connector jar from here:
> >> http://dev.mysql.com/downloads/connector/j/
> >> 2. Run the following command (as user oozie):
> >>           sudo  -u oozie <path-to-oozie-installation/oozie-setup.sh
> >> -jars <path-to-connector-jar>
> >> 3. Create the following DB in your MySQL and grant privs on it:
> >>             # mysql -u root -p
> >>            create database oozie;
> >>            grant all privileges on oozie.* to 'oozie'@'localhost'
> >> identified by 'oozie';
> >> 4. Replace the following properties in your oozie-site.xml with the
> >> values give below:
> >>
> >>     <property>
> >>         <name>oozie.service.StoreService.jdbc.driver</name>
> >>         <value>com.mysql.jdbc.Driver</value>
> >>         <description>
> >>             JDBC driver class.
> >>         </description>
> >>     </property>
> >>
> >>     <property>
> >>         <name>oozie.service.StoreService.jdbc.url</name>
> >>         <value>jdbc:mysql://localhost:3306/oozie</value>
> >>         <description>
> >>             JDBC URL.
> >>         </description>
> >>     </property>
> >>
> >>     <property>
> >>         <name>oozie.service.StoreService.jdbc.username</name>
> >>         <value>oozie</value>
> >>         <description>
> >>             DB user name.
> >>         </description>
> >>     </property>
> >>
> >>     <property>
> >>         <name>oozie.service.StoreService.jdbc.password</name>
> >>         <value>oozie</value>
> >>         <description>
> >>             DB user password.
> >>
> >>             IMPORTANT: if password is emtpy leave a 1 space string, the
> >> service
> >> trims the value,
> >>                        if empty Configuration assumes it is NULL.
> >>
> >>             IMPORTANT: if the StoreServicePasswordService is active,
> >> it will reset
> >> this value with the value given in
> >>                        the console.
> >>         </description>
> >>     </property>
> >>
> >> <-----------------end snip>
> >>
> >>
> >> ---
> >> Gautam
> >>
> >
> >
> >
> > --
> > Harsh J
>



-- 
Alejandro

Re: Oozie Mysql connection

Posted by Gautam Singaraju <ga...@gmail.com>.
Thank you Harsh, for your help. I was able to redo some steps and was
able to start oozie with the MySQL configuration as described.
However, something very weird seems to be happening. Once I schedule a
job, the job completes successfully. But, when I log into MySQL client
to see the schemas, I dont see anything. Then I noticed that I had the
following in the configuration:

<property>
  <name>hive.stats.dbclass</name>
  <value>jdbc:derby</value>
  <description>The default database that stores temporary hive
statistics.</description>
</property>

<property>
  <name>hive.stats.autogather</name>
  <value>true</value>
  <description>A flag to gather statistics automatically during the
INSERT OVERWRITE command.</description>
</property>

<property>
  <name>hive.stats.jdbcdriver</name>
  <value>org.apache.derby.jdbc.EmbeddedDriver</value>
  <description>The JDBC driver for the database that stores temporary
hive statistics.</description>
</property>

<property>
  <name>hive.stats.dbconnectionstring</name>
  <value>jdbc:derby:;databaseName=TempStatsStore;create=true</value>
  <description>The default connection string for the database that
stores temporary hive statistics.</description>
</property>


Could someone explain what is happening here? Does oozie store temp
jobs in a derby db and flush to mysql at a predetermined interval?

Thank you,
---
Gautam


On Thu, Jul 19, 2012 at 3:45 AM, Harsh J <ha...@cloudera.com> wrote:
> Hey Gautam,
>
> Can you also attach a server side oozie.log snippet after running an Oozie
> job that seems to fail with the error you've sent?
>
> I do not think your issue is related to MySQL at this point, but server
> logs will help determine. Thanks!
>
> On Thu, Jul 19, 2012 at 1:09 AM, Gautam Singaraju <
> gautam.singaraju@gmail.com> wrote:
>
>> Hi Oozie community:
>>
>>
>> I am working on getting Oozie working with MySQL; and I have followed
>> the instructions in a previous post:
>>
>> http://mail-archives.apache.org/mod_mbox/incubator-oozie-users/201110.mbox/%3CCAC8XGpEFCTEnHZjdbPQoKgWm-jiK0gzY=A_9WdJGZUzPjEfnAA@mail.gmail.com%3E
>> (I have copied some part of the suggestions at the end of the email.)
>>
>> I am using Oozie 3.2 and mysql 5.1.
>>
>> Exceptions:
>> 1. I have installed oozie as linux user: hadoop.
>> 2. also started it with webconsole.
>> 3. I am running on AWS.
>>
>> When I start oozie, and submit a job, I am getting the following error:
>>
>> WARN AuthenticationFilter:381 - Authentication exception:
>> AuthenticationToken expired
>> org.apache.hadoop.security.authentication.client.AuthenticationException:
>> AuthenticationToken expired
>>
>> Any suggestions would be greatly appreciated. Additionally, Apache
>> Oozie can benefit from a writeup on how to get oozie and mysql
>> working.
>>
>> Thanks in advance!
>> <--------------snip>
>>
>> 1. Get a jdbc connector jar from here:
>> http://dev.mysql.com/downloads/connector/j/
>> 2. Run the following command (as user oozie):
>>           sudo  -u oozie <path-to-oozie-installation/oozie-setup.sh
>> -jars <path-to-connector-jar>
>> 3. Create the following DB in your MySQL and grant privs on it:
>>             # mysql -u root -p
>>            create database oozie;
>>            grant all privileges on oozie.* to 'oozie'@'localhost'
>> identified by 'oozie';
>> 4. Replace the following properties in your oozie-site.xml with the
>> values give below:
>>
>>     <property>
>>         <name>oozie.service.StoreService.jdbc.driver</name>
>>         <value>com.mysql.jdbc.Driver</value>
>>         <description>
>>             JDBC driver class.
>>         </description>
>>     </property>
>>
>>     <property>
>>         <name>oozie.service.StoreService.jdbc.url</name>
>>         <value>jdbc:mysql://localhost:3306/oozie</value>
>>         <description>
>>             JDBC URL.
>>         </description>
>>     </property>
>>
>>     <property>
>>         <name>oozie.service.StoreService.jdbc.username</name>
>>         <value>oozie</value>
>>         <description>
>>             DB user name.
>>         </description>
>>     </property>
>>
>>     <property>
>>         <name>oozie.service.StoreService.jdbc.password</name>
>>         <value>oozie</value>
>>         <description>
>>             DB user password.
>>
>>             IMPORTANT: if password is emtpy leave a 1 space string, the
>> service
>> trims the value,
>>                        if empty Configuration assumes it is NULL.
>>
>>             IMPORTANT: if the StoreServicePasswordService is active,
>> it will reset
>> this value with the value given in
>>                        the console.
>>         </description>
>>     </property>
>>
>> <-----------------end snip>
>>
>>
>> ---
>> Gautam
>>
>
>
>
> --
> Harsh J

Re: Oozie Mysql connection

Posted by Harsh J <ha...@cloudera.com>.
Hey Gautam,

Can you also attach a server side oozie.log snippet after running an Oozie
job that seems to fail with the error you've sent?

I do not think your issue is related to MySQL at this point, but server
logs will help determine. Thanks!

On Thu, Jul 19, 2012 at 1:09 AM, Gautam Singaraju <
gautam.singaraju@gmail.com> wrote:

> Hi Oozie community:
>
>
> I am working on getting Oozie working with MySQL; and I have followed
> the instructions in a previous post:
>
> http://mail-archives.apache.org/mod_mbox/incubator-oozie-users/201110.mbox/%3CCAC8XGpEFCTEnHZjdbPQoKgWm-jiK0gzY=A_9WdJGZUzPjEfnAA@mail.gmail.com%3E
> (I have copied some part of the suggestions at the end of the email.)
>
> I am using Oozie 3.2 and mysql 5.1.
>
> Exceptions:
> 1. I have installed oozie as linux user: hadoop.
> 2. also started it with webconsole.
> 3. I am running on AWS.
>
> When I start oozie, and submit a job, I am getting the following error:
>
> WARN AuthenticationFilter:381 - Authentication exception:
> AuthenticationToken expired
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> AuthenticationToken expired
>
> Any suggestions would be greatly appreciated. Additionally, Apache
> Oozie can benefit from a writeup on how to get oozie and mysql
> working.
>
> Thanks in advance!
> <--------------snip>
>
> 1. Get a jdbc connector jar from here:
> http://dev.mysql.com/downloads/connector/j/
> 2. Run the following command (as user oozie):
>           sudo  -u oozie <path-to-oozie-installation/oozie-setup.sh
> -jars <path-to-connector-jar>
> 3. Create the following DB in your MySQL and grant privs on it:
>             # mysql -u root -p
>            create database oozie;
>            grant all privileges on oozie.* to 'oozie'@'localhost'
> identified by 'oozie';
> 4. Replace the following properties in your oozie-site.xml with the
> values give below:
>
>     <property>
>         <name>oozie.service.StoreService.jdbc.driver</name>
>         <value>com.mysql.jdbc.Driver</value>
>         <description>
>             JDBC driver class.
>         </description>
>     </property>
>
>     <property>
>         <name>oozie.service.StoreService.jdbc.url</name>
>         <value>jdbc:mysql://localhost:3306/oozie</value>
>         <description>
>             JDBC URL.
>         </description>
>     </property>
>
>     <property>
>         <name>oozie.service.StoreService.jdbc.username</name>
>         <value>oozie</value>
>         <description>
>             DB user name.
>         </description>
>     </property>
>
>     <property>
>         <name>oozie.service.StoreService.jdbc.password</name>
>         <value>oozie</value>
>         <description>
>             DB user password.
>
>             IMPORTANT: if password is emtpy leave a 1 space string, the
> service
> trims the value,
>                        if empty Configuration assumes it is NULL.
>
>             IMPORTANT: if the StoreServicePasswordService is active,
> it will reset
> this value with the value given in
>                        the console.
>         </description>
>     </property>
>
> <-----------------end snip>
>
>
> ---
> Gautam
>



-- 
Harsh J