You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Mithun Radhakrishnan (Created) (JIRA)" <ji...@apache.org> on 2012/04/06 02:56:21 UTC
[jira] [Created] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
---------------------------------------------------------------------------------
Key: HIVE-2928
URL: https://issues.apache.org/jira/browse/HIVE-2928
Project: Hive
Issue Type: New Feature
Components: Metastore
Affects Versions: 0.8.1
Reporter: Mithun Radhakrishnan
I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
<quote>
Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
(datastore), but type expected was LONGVARCHAR (metadata). Please check that
the type in the datastore and the type specified in the MetaData are
consistent.
org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
(datastore), but type expected was LONGVARCHAR (metadata). Please check that
the type in the datastore and the type specified in the MetaData are
consistent.
at
org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
at
org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
</quote>
But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
3. The unit-tests seem to run through.
Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
Mithun
P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2928:
---------------------------------
Fix Version/s: 0.9.1
Backported to 0.9.1
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Assignee: Mithun Radhakrishnan
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2928-fixed-path.diff.txt, HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Mithun Radhakrishnan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mithun Radhakrishnan updated HIVE-2928:
---------------------------------------
Attachment: (was: HIVE-2928.patch)
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290826#comment-13290826 ]
Hudson commented on HIVE-2928:
------------------------------
Integrated in Hive-0.9.1-SNAPSHOT-h0.21 #40 (See [https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/40/])
HIVE-2928. Support for Oracle-backed Hive-Metastore (longvarchar to clob in package.jdo) (Revision 1347397)
Result = FAILURE
cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347397
Files :
* /hive/branches/branch-0.9/metastore/scripts/upgrade/oracle
* /hive/branches/branch-0.9/metastore/scripts/upgrade/oracle/hive-schema-0.9.0.oracle.sql
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Assignee: Mithun Radhakrishnan
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2928-fixed-path.diff.txt, HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260135#comment-13260135 ]
Hudson commented on HIVE-2928:
------------------------------
Integrated in Hive-trunk-h0.21 #1391 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1391/])
HIVE-2928. Support for Oracle-backed Hive-Metastore (longvarchar to clob in package.jdo) (Mithun Radhakrishnan and Andrew Bayer via cws) (Revision 1329416)
Result = FAILURE
cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329416
Files :
* /hive/trunk/metastore/scripts/upgrade/oracle
* /hive/trunk/metastore/scripts/upgrade/oracle/hive-schema-0.9.0.oracle.sql
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Assignee: Mithun Radhakrishnan
> Fix For: 0.10.0
>
> Attachments: HIVE-2928-fixed-path.diff.txt, HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Mithun Radhakrishnan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mithun Radhakrishnan updated HIVE-2928:
---------------------------------------
Attachment: HIVE-2928.patch
Updated to remove package.jdo change.
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251979#comment-13251979 ]
Carl Steinbach commented on HIVE-2928:
--------------------------------------
@Mithun: That's great news! Thanks for experimenting with this.
Is the patch ready for review? If so can you please click the "submit patch" button? Thanks.
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2928:
-----------------------------------
Fix Version/s: (was: 0.9.0)
Unlinking from 0.9
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Assignee: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Mithun Radhakrishnan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250786#comment-13250786 ]
Mithun Radhakrishnan commented on HIVE-2928:
--------------------------------------------
@Carl: Thank you for the pointer.
Would your suggestion be that changing the datatype in Hive's pacakge.jdo won't be acceptable because there might be deployments with MySQL with JDO column-validations turned on?
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Mithun Radhakrishnan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253917#comment-13253917 ]
Mithun Radhakrishnan commented on HIVE-2928:
--------------------------------------------
Hey, Carl. I have submitted the patch. I'd appreciate it if you'd look over the Oracle-schema file. (It's pretty straightforward.)
(Thanks very much for looking at this. Much appreciated.)
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Assignee: Mithun Radhakrishnan
> Fix For: 0.9.0
>
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Mithun Radhakrishnan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mithun Radhakrishnan updated HIVE-2928:
---------------------------------------
Affects Version/s: (was: 0.8.1)
0.9.0
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Ashutosh Chauhan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249312#comment-13249312 ]
Ashutosh Chauhan commented on HIVE-2928:
----------------------------------------
bq. 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
You are changing type of columns. This sounds like requiring data migration and thus data migration and schema upgrade scripts for mysql and derby.
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290839#comment-13290839 ]
Hudson commented on HIVE-2928:
------------------------------
Integrated in Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #40 (See [https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/40/])
HIVE-2928. Support for Oracle-backed Hive-Metastore (longvarchar to clob in package.jdo) (Revision 1347397)
Result = FAILURE
cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347397
Files :
* /hive/branches/branch-0.9/metastore/scripts/upgrade/oracle
* /hive/branches/branch-0.9/metastore/scripts/upgrade/oracle/hive-schema-0.9.0.oracle.sql
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Assignee: Mithun Radhakrishnan
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2928-fixed-path.diff.txt, HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Mithun Radhakrishnan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mithun Radhakrishnan updated HIVE-2928:
---------------------------------------
Fix Version/s: 0.9.0
Status: Patch Available (was: Open)
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Fix For: 0.9.0
>
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2928:
---------------------------------
Resolution: Fixed
Fix Version/s: 0.10.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Mithun and Andrew!
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Assignee: Mithun Radhakrishnan
> Fix For: 0.10.0
>
> Attachments: HIVE-2928-fixed-path.diff.txt, HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Mithun Radhakrishnan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mithun Radhakrishnan updated HIVE-2928:
---------------------------------------
Attachment: HIVE-2928.patch
Schema-creation in Oracle. + Changes to package.jdo ("longvarchar" to "clob").
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Mithun Radhakrishnan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250145#comment-13250145 ]
Mithun Radhakrishnan commented on HIVE-2928:
--------------------------------------------
(Just re-verified that this is the case.)
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Andrew Bayer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Bayer updated HIVE-2928:
-------------------------------
Attachment: HIVE-2928-fixed-path.diff.txt
Fixed path of file.
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Assignee: Mithun Radhakrishnan
> Attachments: HIVE-2928-fixed-path.diff.txt, HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Ashutosh Chauhan (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan reassigned HIVE-2928:
--------------------------------------
Assignee: Mithun Radhakrishnan
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Assignee: Mithun Radhakrishnan
> Fix For: 0.9.0
>
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250361#comment-13250361 ]
Carl Steinbach commented on HIVE-2928:
--------------------------------------
@Mithun: What happens on MySQL if you turn column validation on (e.g. datanucleus.validateColumns=true at startup)? There's a note on the Datanucleus site that indicates this won't work:
http://www.datanucleus.org/products/datanucleus/rdbms/support.html
{quote}
You can specify "BLOB", "CLOB" JDBC types when using MySQL with DataNucleus but you must turn validation of columns OFF. This is because these types are not supported by the MySQL JDBC driver and it returns them as LONGVARBINARY/LONGVARCHAR when querying the column type.
{quote}
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Mithun Radhakrishnan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251965#comment-13251965 ]
Mithun Radhakrishnan commented on HIVE-2928:
--------------------------------------------
@Carl: Thanks. I'd seen that thread before, but I hadn't actually tested that out myself. I just did, and here's what I found out:
1. If the complete schema is created apriori (using the hive-schema-0.9.0.oracle.sql in the attached patch), then hive works with Oracle. (This is with "datanucleus.validateColumns = false".)
2. What's neat is that "datanucleus.autoCreateSchema = true" doesn't mess this up, because the schema is completely constructed. The exception in the Original Description was a result of there being schema differences that JDO attempted to resolve.
(For the record, Datanucleus does recommend turning both flags off, for the sake of performance. They're meant to be used for the first start-up.)
The offshoot of this would be that any changes in the schema would have to be resolved using a migration-script for Oracle, and won't be done automatically by the JDO-lib.
I've modified the attached patch to remove the proposed package.jdo change, and keep just the Oracle schema-sql script.
Thanks a bunch.
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Mithun Radhakrishnan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249946#comment-13249946 ]
Mithun Radhakrishnan commented on HIVE-2928:
--------------------------------------------
@Ashutosh: The data-type change is in the package.jdo. I've verified that hive-jars built with CLOB work against a pre-existing MySQL metastore-server (whose datatypes haven't been changed). I didn't have to migrate.
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed
Hive-Metastore ("longvarchar" to "clob" in package.jdo)
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250981#comment-13250981 ]
Carl Steinbach commented on HIVE-2928:
--------------------------------------
We currently have JDO schema validation disabled by default since it affects performance. However, if modifying Hive's package.jdo to work with Oracle means that we're forfeiting this feature, I'd at least like to know that up front.
Also, there was an exchange a couple years ago on the hive-user list where John made an interesting suggestion:
http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/201006.mbox/%3CBD1FE08F-5EDF-4D90-A741-8B703CD06BC1@facebook.com%3E
bq. Another option is to precreate your schema in Oracle and then tell JDO not to try to create/update
it automatically.
Would you mind trying this out to see if it works? If it does then I think that might be the optimal solution for now.
> Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2928
> URL: https://issues.apache.org/jira/browse/HIVE-2928
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Affects Versions: 0.9.0
> Reporter: Mithun Radhakrishnan
> Attachments: HIVE-2928.patch
>
>
> I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on.
> One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception:
> <quote>
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
> Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
> (datastore), but type expected was LONGVARCHAR (metadata). Please check that
> the type in the datastore and the type specified in the MetaData are
> consistent.
> at
> org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
> at
> org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
> </quote>
> But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising:
> 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.)
> 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry.
> 3. The unit-tests seem to run through.
> Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle?
> Mithun
> P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira