You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Ryota Egashira <eg...@yahoo-inc.com> on 2013/03/01 23:52:34 UTC

Re: Review Request: [OOZIE-1150 (2)] DB upgrade scripts for hcat changes

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9534/
-----------------------------------------------------------

(Updated March 1, 2013, 10:52 p.m.)


Review request for oozie.


Changes
-------

fixed tucu's comments, and did upgrade test(from 3.1.*, and 3.3.*) using mysql(5.0.96), postgres (9.1.8), oracle (11.2.0.2.0)
 


Description
-------

OOZIE-1150
note this is extending existing Rohini's work, https://reviews.apache.org/r/8770/
added change on OozieDBCLI to support different types of DB.


This addresses bug OOZIE-1150.
    https://issues.apache.org/jira/browse/OOZIE-1150


Diffs (updated)
-----

  branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1451135 

Diff: https://reviews.apache.org/r/9534/diff/


Testing
-------

locally ran OOZIE on top of MYSQL, and checked DB schema and ran examples sucessfully


Thanks,

Ryota Egashira


Re: Review Request: [OOZIE-1150 (2)] DB upgrade scripts for hcat changes

Posted by Ryota Egashira <eg...@yahoo-inc.com>.

> On April 9, 2013, 4:51 p.m., Alejandro Abdelnur wrote:
> > LGTM. Ryota, please update in the JIRA all DBs and update combinations you've test it.

added comment https://issues.apache.org/jira/browse/OOZIE-1150?focusedCommentId=13627267&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13627267 
thanks!


- Ryota


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9534/#review18858
-----------------------------------------------------------


On April 8, 2013, 8:15 a.m., Ryota Egashira wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9534/
> -----------------------------------------------------------
> 
> (Updated April 8, 2013, 8:15 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Description
> -------
> 
> OOZIE-1150
> note this is extending existing Rohini's work, https://reviews.apache.org/r/8770/
> added change on OozieDBCLI to support different types of DB.
> 
> 
> This addresses bug OOZIE-1150.
>     https://issues.apache.org/jira/browse/OOZIE-1150
> 
> 
> Diffs
> -----
> 
>   branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1451135 
> 
> Diff: https://reviews.apache.org/r/9534/diff/
> 
> 
> Testing
> -------
> 
> locally ran OOZIE on top of MYSQL, and checked DB schema and ran examples sucessfully
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>


Re: Review Request: [OOZIE-1150 (2)] DB upgrade scripts for hcat changes

Posted by Alejandro Abdelnur <tu...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9534/#review18858
-----------------------------------------------------------

Ship it!


LGTM. Ryota, please update in the JIRA all DBs and update combinations you've test it.

- Alejandro Abdelnur


On April 8, 2013, 8:15 a.m., Ryota Egashira wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9534/
> -----------------------------------------------------------
> 
> (Updated April 8, 2013, 8:15 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Description
> -------
> 
> OOZIE-1150
> note this is extending existing Rohini's work, https://reviews.apache.org/r/8770/
> added change on OozieDBCLI to support different types of DB.
> 
> 
> This addresses bug OOZIE-1150.
>     https://issues.apache.org/jira/browse/OOZIE-1150
> 
> 
> Diffs
> -----
> 
>   branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1451135 
> 
> Diff: https://reviews.apache.org/r/9534/diff/
> 
> 
> Testing
> -------
> 
> locally ran OOZIE on top of MYSQL, and checked DB schema and ran examples sucessfully
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>


Re: Review Request: [OOZIE-1150 (2)] DB upgrade scripts for hcat changes

Posted by Ryota Egashira <eg...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9534/
-----------------------------------------------------------

(Updated April 8, 2013, 8:15 a.m.)


Review request for oozie.


Changes
-------

fixed tucu's comments
- for derby, added new method to read row by row and replace string in MISSING_DEPENDENCIES (Derby doesn't support Replace function in SQL, also UDF cannot be applied to CLOB)
- remove SQL scripts (for Oracle) from the patch


Description
-------

OOZIE-1150
note this is extending existing Rohini's work, https://reviews.apache.org/r/8770/
added change on OozieDBCLI to support different types of DB.


This addresses bug OOZIE-1150.
    https://issues.apache.org/jira/browse/OOZIE-1150


Diffs (updated)
-----

  branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1451135 

Diff: https://reviews.apache.org/r/9534/diff/


Testing
-------

locally ran OOZIE on top of MYSQL, and checked DB schema and ran examples sucessfully


Thanks,

Ryota Egashira


Re: Review Request: [OOZIE-1150 (2)] DB upgrade scripts for hcat changes

Posted by Ryota Egashira <eg...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9534/
-----------------------------------------------------------

(Updated March 4, 2013, 10:03 a.m.)


Review request for oozie.


Changes
-------

fixed tucu's comments. ran upgrade test again for confirmation. 


Description
-------

OOZIE-1150
note this is extending existing Rohini's work, https://reviews.apache.org/r/8770/
added change on OozieDBCLI to support different types of DB.


This addresses bug OOZIE-1150.
    https://issues.apache.org/jira/browse/OOZIE-1150


Diffs (updated)
-----

  branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1451135 
  branches/hcat-intre/utils/dbutils/updatescripts/dbscripts-3.2.0.sql PRE-CREATION 
  branches/hcat-intre/utils/dbutils/updatescripts/dbscripts-4.0.0.sql PRE-CREATION 
  branches/hcat-intre/utils/dbutils/updatescripts/updatescripts-3.1.0-to-3.2.0.sql PRE-CREATION 
  branches/hcat-intre/utils/dbutils/updatescripts/updatescripts-3.2.0-to-4.0.0.sql PRE-CREATION 

Diff: https://reviews.apache.org/r/9534/diff/


Testing
-------

locally ran OOZIE on top of MYSQL, and checked DB schema and ran examples sucessfully


Thanks,

Ryota Egashira


Re: Review Request: [OOZIE-1150 (2)] DB upgrade scripts for hcat changes

Posted by Ryota Egashira <eg...@yahoo-inc.com>.

> On March 1, 2013, 11:03 p.m., Alejandro Abdelnur wrote:
> > branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java, line 277
> > <https://reviews.apache.org/r/9534/diff/4/?file=264330#file264330line277>
> >
> >     does this work in ALL DB types?
> 
> Ryota Egashira wrote:
>     oh great point. it worked for mysql/postgres/oracle, but just noticed derby complained. (upgrading in-memory DB is rare compared to other DB, so didn't check as thoroughly, but devil is in there..)
>     will fix it.
> 
> Alejandro Abdelnur wrote:
>     btw, wouldn't this qualify as a post upgrade task to DB VERSION = 2 ?

fixed derby case, and moved the SQL to postUpgradeTasks


> On March 1, 2013, 11:03 p.m., Alejandro Abdelnur wrote:
> > branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java, line 283
> > <https://reviews.apache.org/r/9534/diff/4/?file=264330#file264330line283>
> >
> >     Why do we need all these ALTER TABLE, OpenJPA take care of doing that magic, no?
> 
> Ryota Egashira wrote:
>     not expert of openJPA, so I need to do small experiment to check MappingTool does that or not.

did test and confirmed that MappingTool takes care of it. 
so removing DB-dependent ALTER TABLE statement, and instead call createUpgradeDB.
(code is cleaner after DB-dependent part gone)


- Ryota


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9534/#review17275
-----------------------------------------------------------


On March 1, 2013, 10:52 p.m., Ryota Egashira wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9534/
> -----------------------------------------------------------
> 
> (Updated March 1, 2013, 10:52 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Description
> -------
> 
> OOZIE-1150
> note this is extending existing Rohini's work, https://reviews.apache.org/r/8770/
> added change on OozieDBCLI to support different types of DB.
> 
> 
> This addresses bug OOZIE-1150.
>     https://issues.apache.org/jira/browse/OOZIE-1150
> 
> 
> Diffs
> -----
> 
>   branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1451135 
> 
> Diff: https://reviews.apache.org/r/9534/diff/
> 
> 
> Testing
> -------
> 
> locally ran OOZIE on top of MYSQL, and checked DB schema and ran examples sucessfully
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>


Re: Review Request: [OOZIE-1150 (2)] DB upgrade scripts for hcat changes

Posted by Ryota Egashira <eg...@yahoo-inc.com>.

> On March 1, 2013, 11:03 p.m., Alejandro Abdelnur wrote:
> > branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java, line 277
> > <https://reviews.apache.org/r/9534/diff/4/?file=264330#file264330line277>
> >
> >     does this work in ALL DB types?

oh great point. it worked for mysql/postgres/oracle, but just noticed derby complained. (upgrading in-memory DB is rare compared to other DB, so didn't check as thoroughly, but devil is in there..)
will fix it.


> On March 1, 2013, 11:03 p.m., Alejandro Abdelnur wrote:
> > branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java, line 283
> > <https://reviews.apache.org/r/9534/diff/4/?file=264330#file264330line283>
> >
> >     Why do we need all these ALTER TABLE, OpenJPA take care of doing that magic, no?

not expert of openJPA, so I need to do small experiment to check MappingTool does that or not. 


- Ryota


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9534/#review17275
-----------------------------------------------------------


On March 1, 2013, 10:52 p.m., Ryota Egashira wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9534/
> -----------------------------------------------------------
> 
> (Updated March 1, 2013, 10:52 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Description
> -------
> 
> OOZIE-1150
> note this is extending existing Rohini's work, https://reviews.apache.org/r/8770/
> added change on OozieDBCLI to support different types of DB.
> 
> 
> This addresses bug OOZIE-1150.
>     https://issues.apache.org/jira/browse/OOZIE-1150
> 
> 
> Diffs
> -----
> 
>   branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1451135 
> 
> Diff: https://reviews.apache.org/r/9534/diff/
> 
> 
> Testing
> -------
> 
> locally ran OOZIE on top of MYSQL, and checked DB schema and ran examples sucessfully
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>


Re: Review Request: [OOZIE-1150 (2)] DB upgrade scripts for hcat changes

Posted by Alejandro Abdelnur <tu...@cloudera.com>.

> On March 1, 2013, 11:03 p.m., Alejandro Abdelnur wrote:
> > branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java, line 277
> > <https://reviews.apache.org/r/9534/diff/4/?file=264330#file264330line277>
> >
> >     does this work in ALL DB types?
> 
> Ryota Egashira wrote:
>     oh great point. it worked for mysql/postgres/oracle, but just noticed derby complained. (upgrading in-memory DB is rare compared to other DB, so didn't check as thoroughly, but devil is in there..)
>     will fix it.

btw, wouldn't this qualify as a post upgrade task to DB VERSION = 2 ?


- Alejandro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9534/#review17275
-----------------------------------------------------------


On March 1, 2013, 10:52 p.m., Ryota Egashira wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9534/
> -----------------------------------------------------------
> 
> (Updated March 1, 2013, 10:52 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Description
> -------
> 
> OOZIE-1150
> note this is extending existing Rohini's work, https://reviews.apache.org/r/8770/
> added change on OozieDBCLI to support different types of DB.
> 
> 
> This addresses bug OOZIE-1150.
>     https://issues.apache.org/jira/browse/OOZIE-1150
> 
> 
> Diffs
> -----
> 
>   branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1451135 
> 
> Diff: https://reviews.apache.org/r/9534/diff/
> 
> 
> Testing
> -------
> 
> locally ran OOZIE on top of MYSQL, and checked DB schema and ran examples sucessfully
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>


Re: Review Request: [OOZIE-1150 (2)] DB upgrade scripts for hcat changes

Posted by Alejandro Abdelnur <tu...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9534/#review17275
-----------------------------------------------------------



branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java
<https://reviews.apache.org/r/9534/#comment36742>

    does this work in ALL DB types?



branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java
<https://reviews.apache.org/r/9534/#comment36741>

    Why do we need all these ALTER TABLE, OpenJPA take care of doing that magic, no? 


- Alejandro Abdelnur


On March 1, 2013, 10:52 p.m., Ryota Egashira wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9534/
> -----------------------------------------------------------
> 
> (Updated March 1, 2013, 10:52 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Description
> -------
> 
> OOZIE-1150
> note this is extending existing Rohini's work, https://reviews.apache.org/r/8770/
> added change on OozieDBCLI to support different types of DB.
> 
> 
> This addresses bug OOZIE-1150.
>     https://issues.apache.org/jira/browse/OOZIE-1150
> 
> 
> Diffs
> -----
> 
>   branches/hcat-intre/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1451135 
> 
> Diff: https://reviews.apache.org/r/9534/diff/
> 
> 
> Testing
> -------
> 
> locally ran OOZIE on top of MYSQL, and checked DB schema and ran examples sucessfully
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>