You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oodt.apache.org by "Chris A. Mattmann (Created) (JIRA)" <ji...@apache.org> on 2012/03/15 00:38:39 UTC
[jira] [Created] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
DataSourceCatalog compatible with HypersonicSQL
-----------------------------------------------
Key: OODT-410
URL: https://issues.apache.org/jira/browse/OODT-410
Project: OODT
Issue Type: New Feature
Components: file manager
Reporter: Chris A. Mattmann
Assignee: Chris A. Mattmann
Fix For: 0.4
This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231200#comment-13231200 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-15 00:35:26, brian Foster wrote:
bq. > Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, great question. The reality is that this catalog was hidden away in the File Manager test suite inside of TestDataSourceCatalog. There was something wiggy about the DataSourceCatalog code that Hypersonic SQL didn't like so I had to extend DataSourceCatalog for internal testing since it made sense to use Hypersonic SQL for that (could load up the DBMS in memory, etc.) I know that there was like 1 method that used incompatible SQL syntax that Hypersonic SQL didn't like. Then when I overrode that one method, I had to override like 4-5 others to inject my subclass method into the call chain for that which is what you see in the patch. It seems that DataSourceCatalog has since evolved and it's entirely possible that it would support Hypersonic SQL now, so here's what I see as 2 options:
bq.
bq. 1. we try and simply remove Hypersonic SQL catalog period, and then see if the TestDataSourceCatalog passes. If it does, this issue is moot and I will close this ReviewBoard and the issue.
bq.
bq. 2. If #1 doesn't succeed, and suggests changes, we could:
bq. a) flow those changes into DataSourceCatalog -- someone would have to have time to do that :) It also could happen later; or
bq. b) suggest that since this catalog exists and we know it works, temporarily in 0.4 and ongoing until someone refactors or improves DataSourceCatalog ship the HypersonicSQLFriendlyCatalog for folks who want to use it.
bq.
bq. Sound right? Thoughts?
bq.
bq. brian Foster wrote:
bq. i'd be cool with you checking it in for now with a TODO at the top of the class and maybe a JIRA issue to merge the 2 at some point
bq.
bq. Chris Mattmann wrote:
bq. Cool, I will try and use the DataSourceCatalog, see if it passes, and if so, just forget this one. If it doesn't pass, I'll consider the above your +1 and then move forward with this.
bq.
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, here was the issue, check it:
bq.
bq. java.sql.SQLException: Not in aggregate function or group by clause: org.hsqldb.Expression@16ba5c7a in statement [SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p ORDER BY p.product_id DESC ]
bq. at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.executeQuery(Unknown Source)
bq.
bq. Hypersonic SQL doesn't like COUNT functions and ORDER BY to be used together. See: http://www.dcm4che.org/jira/browse/ARR-37
bq.
bq. So, what I did with this catalog is override the getResultListSize, which then caused me to have to override paginateQuery, and then caused me to have to intercept the pagedQuery and then the full paging API, which is what this catalog does, and I think correctly. If I don't hear any further objections, I am going to commit this tomorrow.
bq.
bq. Ricky Nguyen wrote:
bq. Why can't we remove the ORDER clause?
bq.
bq. SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p
bq.
bq. since the order doesn't matter when you're just returning a count.
bq.
bq. Chris Mattmann wrote:
bq. What do you guys think? I'd be OK removing the ORDER clause but it will be a change from the SQL that the DataSourceCatalog used to generate. But, you're right, it doesn't really matter. Wonder why Oracle, and MySQL and Postgres allow it?
bq.
bq. brian Foster wrote:
bq. I vote ditch ORDER!
bq.
bq. Chris Mattmann wrote:
bq. Hmm VERY interesting. So, I tried to ditch ORDER, and just use DataSourceCatalog. Here's what I got:
bq.
bq. [chipotle:oodt/trunk/filemgr] mattmann% more target/surefire-reports/org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog.txt
bq. -------------------------------------------------------------------------------
bq. Test set: org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog
bq. -------------------------------------------------------------------------------
bq. Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.828 sec <<< FAILURE!
bq. testFirstProductOnlyOnFirstPage(org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog) Time elapsed: 0.124 sec <<< FAILURE!
bq. junit.framework.ComparisonFailure: expected:<[ShouldBeFirstForPage.tx]t> but was:<[tes]t>
bq. at junit.framework.Assert.assertEquals(Assert.java:81)
bq. at junit.framework.Assert.assertEquals(Assert.java:87)
bq. at org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog.testFirstProductOnlyOnFirstPage(TestDataSourceCatalog.java:278)
bq.
bq. [chipotle:oodt/trunk/filemgr] mattmann%
bq.
bq. I need to look more, but something is wiggy here.
bq.
bq. brian Foster wrote:
bq. When i ditch DataSourceCatalog line number 1793:
bq.
bq. getProductSql += "ORDER BY p.product_id DESC ";
bq.
bq. TestDataSourceCatalog pass for me
I'd vote to kick ORDER sorts when not needed; it just adds time to the query execution.
- Paul
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
On 2012-03-14 23:40:55, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 23:40:55)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230841#comment-13230841 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-15 00:35:26, brian Foster wrote:
bq. > Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, great question. The reality is that this catalog was hidden away in the File Manager test suite inside of TestDataSourceCatalog. There was something wiggy about the DataSourceCatalog code that Hypersonic SQL didn't like so I had to extend DataSourceCatalog for internal testing since it made sense to use Hypersonic SQL for that (could load up the DBMS in memory, etc.) I know that there was like 1 method that used incompatible SQL syntax that Hypersonic SQL didn't like. Then when I overrode that one method, I had to override like 4-5 others to inject my subclass method into the call chain for that which is what you see in the patch. It seems that DataSourceCatalog has since evolved and it's entirely possible that it would support Hypersonic SQL now, so here's what I see as 2 options:
bq.
bq. 1. we try and simply remove Hypersonic SQL catalog period, and then see if the TestDataSourceCatalog passes. If it does, this issue is moot and I will close this ReviewBoard and the issue.
bq.
bq. 2. If #1 doesn't succeed, and suggests changes, we could:
bq. a) flow those changes into DataSourceCatalog -- someone would have to have time to do that :) It also could happen later; or
bq. b) suggest that since this catalog exists and we know it works, temporarily in 0.4 and ongoing until someone refactors or improves DataSourceCatalog ship the HypersonicSQLFriendlyCatalog for folks who want to use it.
bq.
bq. Sound right? Thoughts?
bq.
bq. brian Foster wrote:
bq. i'd be cool with you checking it in for now with a TODO at the top of the class and maybe a JIRA issue to merge the 2 at some point
Cool, I will try and use the DataSourceCatalog, see if it passes, and if so, just forget this one. If it doesn't pass, I'll consider the above your +1 and then move forward with this.
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
On 2012-03-14 23:40:55, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 23:40:55)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229799#comment-13229799 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
- brian
On 2012-03-14 23:40:55, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 23:40:55)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231792#comment-13231792 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/
-----------------------------------------------------------
(Updated 2012-03-17 01:40:37.106062)
Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
Changes
-------
OK guys, I got it working. Thanks for the comments to date, Ricky and Brian et al!
All tests pass for me now.
Summary
-------
DataSourceCatalog compatible with HypersonicSQL
This addresses bug OODT-410.
https://issues.apache.org/jira/browse/OODT-410
Diffs (updated)
-----
./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/DataSourceCatalog.java 1301321
./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1301321
Diff: https://reviews.apache.org/r/4358/diff
Testing
-------
Thanks,
Chris
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230884#comment-13230884 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-15 00:35:26, brian Foster wrote:
bq. > Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, great question. The reality is that this catalog was hidden away in the File Manager test suite inside of TestDataSourceCatalog. There was something wiggy about the DataSourceCatalog code that Hypersonic SQL didn't like so I had to extend DataSourceCatalog for internal testing since it made sense to use Hypersonic SQL for that (could load up the DBMS in memory, etc.) I know that there was like 1 method that used incompatible SQL syntax that Hypersonic SQL didn't like. Then when I overrode that one method, I had to override like 4-5 others to inject my subclass method into the call chain for that which is what you see in the patch. It seems that DataSourceCatalog has since evolved and it's entirely possible that it would support Hypersonic SQL now, so here's what I see as 2 options:
bq.
bq. 1. we try and simply remove Hypersonic SQL catalog period, and then see if the TestDataSourceCatalog passes. If it does, this issue is moot and I will close this ReviewBoard and the issue.
bq.
bq. 2. If #1 doesn't succeed, and suggests changes, we could:
bq. a) flow those changes into DataSourceCatalog -- someone would have to have time to do that :) It also could happen later; or
bq. b) suggest that since this catalog exists and we know it works, temporarily in 0.4 and ongoing until someone refactors or improves DataSourceCatalog ship the HypersonicSQLFriendlyCatalog for folks who want to use it.
bq.
bq. Sound right? Thoughts?
bq.
bq. brian Foster wrote:
bq. i'd be cool with you checking it in for now with a TODO at the top of the class and maybe a JIRA issue to merge the 2 at some point
bq.
bq. Chris Mattmann wrote:
bq. Cool, I will try and use the DataSourceCatalog, see if it passes, and if so, just forget this one. If it doesn't pass, I'll consider the above your +1 and then move forward with this.
bq.
Hey Brian, here was the issue, check it:
java.sql.SQLException: Not in aggregate function or group by clause: org.hsqldb.Expression@16ba5c7a in statement [SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p ORDER BY p.product_id DESC ]
at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
at org.hsqldb.jdbc.jdbcStatement.executeQuery(Unknown Source)
Hypersonic SQL doesn't like COUNT functions and ORDER BY to be used together. See: http://www.dcm4che.org/jira/browse/ARR-37
So, what I did with this catalog is override the getResultListSize, which then caused me to have to override paginateQuery, and then caused me to have to intercept the pagedQuery and then the full paging API, which is what this catalog does, and I think correctly. If I don't hear any further objections, I am going to commit this tomorrow.
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
On 2012-03-14 23:40:55, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 23:40:55)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230838#comment-13230838 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-15 00:35:26, brian Foster wrote:
bq. > Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, great question. The reality is that this catalog was hidden away in the File Manager test suite inside of TestDataSourceCatalog. There was something wiggy about the DataSourceCatalog code that Hypersonic SQL didn't like so I had to extend DataSourceCatalog for internal testing since it made sense to use Hypersonic SQL for that (could load up the DBMS in memory, etc.) I know that there was like 1 method that used incompatible SQL syntax that Hypersonic SQL didn't like. Then when I overrode that one method, I had to override like 4-5 others to inject my subclass method into the call chain for that which is what you see in the patch. It seems that DataSourceCatalog has since evolved and it's entirely possible that it would support Hypersonic SQL now, so here's what I see as 2 options:
bq.
bq. 1. we try and simply remove Hypersonic SQL catalog period, and then see if the TestDataSourceCatalog passes. If it does, this issue is moot and I will close this ReviewBoard and the issue.
bq.
bq. 2. If #1 doesn't succeed, and suggests changes, we could:
bq. a) flow those changes into DataSourceCatalog -- someone would have to have time to do that :) It also could happen later; or
bq. b) suggest that since this catalog exists and we know it works, temporarily in 0.4 and ongoing until someone refactors or improves DataSourceCatalog ship the HypersonicSQLFriendlyCatalog for folks who want to use it.
bq.
bq. Sound right? Thoughts?
i'd be cool with you checking it in for now with a TODO at the top of the class and maybe a JIRA issue to merge the 2 at some point
- brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
On 2012-03-14 23:40:55, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 23:40:55)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231208#comment-13231208 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/
-----------------------------------------------------------
(Updated 2012-03-16 14:14:17.061621)
Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
Changes
-------
- Updated diff to not include specific HypersonicSQL catalog and to update by removing ORDER BY from the getResultListSize in DataSourceCatalog. Tests don't pass for me.
Summary
-------
DataSourceCatalog compatible with HypersonicSQL
This addresses bug OODT-410.
https://issues.apache.org/jira/browse/OODT-410
Diffs (updated)
-----
./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/DataSourceCatalog.java 1301321
./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1301321
Diff: https://reviews.apache.org/r/4358/diff
Testing
-------
Thanks,
Chris
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230933#comment-13230933 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-15 00:35:26, brian Foster wrote:
bq. > Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, great question. The reality is that this catalog was hidden away in the File Manager test suite inside of TestDataSourceCatalog. There was something wiggy about the DataSourceCatalog code that Hypersonic SQL didn't like so I had to extend DataSourceCatalog for internal testing since it made sense to use Hypersonic SQL for that (could load up the DBMS in memory, etc.) I know that there was like 1 method that used incompatible SQL syntax that Hypersonic SQL didn't like. Then when I overrode that one method, I had to override like 4-5 others to inject my subclass method into the call chain for that which is what you see in the patch. It seems that DataSourceCatalog has since evolved and it's entirely possible that it would support Hypersonic SQL now, so here's what I see as 2 options:
bq.
bq. 1. we try and simply remove Hypersonic SQL catalog period, and then see if the TestDataSourceCatalog passes. If it does, this issue is moot and I will close this ReviewBoard and the issue.
bq.
bq. 2. If #1 doesn't succeed, and suggests changes, we could:
bq. a) flow those changes into DataSourceCatalog -- someone would have to have time to do that :) It also could happen later; or
bq. b) suggest that since this catalog exists and we know it works, temporarily in 0.4 and ongoing until someone refactors or improves DataSourceCatalog ship the HypersonicSQLFriendlyCatalog for folks who want to use it.
bq.
bq. Sound right? Thoughts?
bq.
bq. brian Foster wrote:
bq. i'd be cool with you checking it in for now with a TODO at the top of the class and maybe a JIRA issue to merge the 2 at some point
bq.
bq. Chris Mattmann wrote:
bq. Cool, I will try and use the DataSourceCatalog, see if it passes, and if so, just forget this one. If it doesn't pass, I'll consider the above your +1 and then move forward with this.
bq.
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, here was the issue, check it:
bq.
bq. java.sql.SQLException: Not in aggregate function or group by clause: org.hsqldb.Expression@16ba5c7a in statement [SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p ORDER BY p.product_id DESC ]
bq. at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.executeQuery(Unknown Source)
bq.
bq. Hypersonic SQL doesn't like COUNT functions and ORDER BY to be used together. See: http://www.dcm4che.org/jira/browse/ARR-37
bq.
bq. So, what I did with this catalog is override the getResultListSize, which then caused me to have to override paginateQuery, and then caused me to have to intercept the pagedQuery and then the full paging API, which is what this catalog does, and I think correctly. If I don't hear any further objections, I am going to commit this tomorrow.
bq.
bq. Ricky Nguyen wrote:
bq. Why can't we remove the ORDER clause?
bq.
bq. SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p
bq.
bq. since the order doesn't matter when you're just returning a count.
bq.
bq. Chris Mattmann wrote:
bq. What do you guys think? I'd be OK removing the ORDER clause but it will be a change from the SQL that the DataSourceCatalog used to generate. But, you're right, it doesn't really matter. Wonder why Oracle, and MySQL and Postgres allow it?
bq.
bq. brian Foster wrote:
bq. I vote ditch ORDER!
bq.
bq. Chris Mattmann wrote:
bq. Hmm VERY interesting. So, I tried to ditch ORDER, and just use DataSourceCatalog. Here's what I got:
bq.
bq. [chipotle:oodt/trunk/filemgr] mattmann% more target/surefire-reports/org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog.txt
bq. -------------------------------------------------------------------------------
bq. Test set: org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog
bq. -------------------------------------------------------------------------------
bq. Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.828 sec <<< FAILURE!
bq. testFirstProductOnlyOnFirstPage(org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog) Time elapsed: 0.124 sec <<< FAILURE!
bq. junit.framework.ComparisonFailure: expected:<[ShouldBeFirstForPage.tx]t> but was:<[tes]t>
bq. at junit.framework.Assert.assertEquals(Assert.java:81)
bq. at junit.framework.Assert.assertEquals(Assert.java:87)
bq. at org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog.testFirstProductOnlyOnFirstPage(TestDataSourceCatalog.java:278)
bq.
bq. [chipotle:oodt/trunk/filemgr] mattmann%
bq.
bq. I need to look more, but something is wiggy here.
When i ditch DataSourceCatalog line number 1793:
getProductSql += "ORDER BY p.product_id DESC ";
TestDataSourceCatalog pass for me
- brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
On 2012-03-14 23:40:55, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 23:40:55)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230920#comment-13230920 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-15 00:35:26, brian Foster wrote:
bq. > Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, great question. The reality is that this catalog was hidden away in the File Manager test suite inside of TestDataSourceCatalog. There was something wiggy about the DataSourceCatalog code that Hypersonic SQL didn't like so I had to extend DataSourceCatalog for internal testing since it made sense to use Hypersonic SQL for that (could load up the DBMS in memory, etc.) I know that there was like 1 method that used incompatible SQL syntax that Hypersonic SQL didn't like. Then when I overrode that one method, I had to override like 4-5 others to inject my subclass method into the call chain for that which is what you see in the patch. It seems that DataSourceCatalog has since evolved and it's entirely possible that it would support Hypersonic SQL now, so here's what I see as 2 options:
bq.
bq. 1. we try and simply remove Hypersonic SQL catalog period, and then see if the TestDataSourceCatalog passes. If it does, this issue is moot and I will close this ReviewBoard and the issue.
bq.
bq. 2. If #1 doesn't succeed, and suggests changes, we could:
bq. a) flow those changes into DataSourceCatalog -- someone would have to have time to do that :) It also could happen later; or
bq. b) suggest that since this catalog exists and we know it works, temporarily in 0.4 and ongoing until someone refactors or improves DataSourceCatalog ship the HypersonicSQLFriendlyCatalog for folks who want to use it.
bq.
bq. Sound right? Thoughts?
bq.
bq. brian Foster wrote:
bq. i'd be cool with you checking it in for now with a TODO at the top of the class and maybe a JIRA issue to merge the 2 at some point
bq.
bq. Chris Mattmann wrote:
bq. Cool, I will try and use the DataSourceCatalog, see if it passes, and if so, just forget this one. If it doesn't pass, I'll consider the above your +1 and then move forward with this.
bq.
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, here was the issue, check it:
bq.
bq. java.sql.SQLException: Not in aggregate function or group by clause: org.hsqldb.Expression@16ba5c7a in statement [SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p ORDER BY p.product_id DESC ]
bq. at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.executeQuery(Unknown Source)
bq.
bq. Hypersonic SQL doesn't like COUNT functions and ORDER BY to be used together. See: http://www.dcm4che.org/jira/browse/ARR-37
bq.
bq. So, what I did with this catalog is override the getResultListSize, which then caused me to have to override paginateQuery, and then caused me to have to intercept the pagedQuery and then the full paging API, which is what this catalog does, and I think correctly. If I don't hear any further objections, I am going to commit this tomorrow.
bq.
bq. Ricky Nguyen wrote:
bq. Why can't we remove the ORDER clause?
bq.
bq. SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p
bq.
bq. since the order doesn't matter when you're just returning a count.
bq.
bq. Chris Mattmann wrote:
bq. What do you guys think? I'd be OK removing the ORDER clause but it will be a change from the SQL that the DataSourceCatalog used to generate. But, you're right, it doesn't really matter. Wonder why Oracle, and MySQL and Postgres allow it?
I vote ditch ORDER!
- brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
On 2012-03-14 23:40:55, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 23:40:55)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230915#comment-13230915 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-15 00:35:26, brian Foster wrote:
bq. > Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, great question. The reality is that this catalog was hidden away in the File Manager test suite inside of TestDataSourceCatalog. There was something wiggy about the DataSourceCatalog code that Hypersonic SQL didn't like so I had to extend DataSourceCatalog for internal testing since it made sense to use Hypersonic SQL for that (could load up the DBMS in memory, etc.) I know that there was like 1 method that used incompatible SQL syntax that Hypersonic SQL didn't like. Then when I overrode that one method, I had to override like 4-5 others to inject my subclass method into the call chain for that which is what you see in the patch. It seems that DataSourceCatalog has since evolved and it's entirely possible that it would support Hypersonic SQL now, so here's what I see as 2 options:
bq.
bq. 1. we try and simply remove Hypersonic SQL catalog period, and then see if the TestDataSourceCatalog passes. If it does, this issue is moot and I will close this ReviewBoard and the issue.
bq.
bq. 2. If #1 doesn't succeed, and suggests changes, we could:
bq. a) flow those changes into DataSourceCatalog -- someone would have to have time to do that :) It also could happen later; or
bq. b) suggest that since this catalog exists and we know it works, temporarily in 0.4 and ongoing until someone refactors or improves DataSourceCatalog ship the HypersonicSQLFriendlyCatalog for folks who want to use it.
bq.
bq. Sound right? Thoughts?
bq.
bq. brian Foster wrote:
bq. i'd be cool with you checking it in for now with a TODO at the top of the class and maybe a JIRA issue to merge the 2 at some point
bq.
bq. Chris Mattmann wrote:
bq. Cool, I will try and use the DataSourceCatalog, see if it passes, and if so, just forget this one. If it doesn't pass, I'll consider the above your +1 and then move forward with this.
bq.
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, here was the issue, check it:
bq.
bq. java.sql.SQLException: Not in aggregate function or group by clause: org.hsqldb.Expression@16ba5c7a in statement [SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p ORDER BY p.product_id DESC ]
bq. at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.executeQuery(Unknown Source)
bq.
bq. Hypersonic SQL doesn't like COUNT functions and ORDER BY to be used together. See: http://www.dcm4che.org/jira/browse/ARR-37
bq.
bq. So, what I did with this catalog is override the getResultListSize, which then caused me to have to override paginateQuery, and then caused me to have to intercept the pagedQuery and then the full paging API, which is what this catalog does, and I think correctly. If I don't hear any further objections, I am going to commit this tomorrow.
bq.
bq. Ricky Nguyen wrote:
bq. Why can't we remove the ORDER clause?
bq.
bq. SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p
bq.
bq. since the order doesn't matter when you're just returning a count.
What do you guys think? I'd be OK removing the ORDER clause but it will be a change from the SQL that the DataSourceCatalog used to generate. But, you're right, it doesn't really matter. Wonder why Oracle, and MySQL and Postgres allow it?
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
On 2012-03-14 23:40:55, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 23:40:55)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231377#comment-13231377 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review6043
-----------------------------------------------------------
./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java
<https://reviews.apache.org/r/4358/#comment13016>
I think HsqlDbFriendlyDataSourceCatalog.paginateQuery was missing an "ORDER BY ... DESC" clause here, whereas DataSourceCatalog (line 1960) does a descending sort.
Since the primary key product_id is incremented (src/testdata/testcat.sql), and "ShouldBeFirstForPage.txt" is inserted LAST, DataSourceCatalog returns "ShouldBeFirstForPage.txt" as the first item on the FIRST page when ordered DESC.
To me, it seems like DataSourceCatalog works properly and the test case needs to be modified.
- Ricky
On 2012-03-16 14:14:17, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-16 14:14:17)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/DataSourceCatalog.java 1301321
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1301321
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230925#comment-13230925 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-15 00:35:26, brian Foster wrote:
bq. > Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, great question. The reality is that this catalog was hidden away in the File Manager test suite inside of TestDataSourceCatalog. There was something wiggy about the DataSourceCatalog code that Hypersonic SQL didn't like so I had to extend DataSourceCatalog for internal testing since it made sense to use Hypersonic SQL for that (could load up the DBMS in memory, etc.) I know that there was like 1 method that used incompatible SQL syntax that Hypersonic SQL didn't like. Then when I overrode that one method, I had to override like 4-5 others to inject my subclass method into the call chain for that which is what you see in the patch. It seems that DataSourceCatalog has since evolved and it's entirely possible that it would support Hypersonic SQL now, so here's what I see as 2 options:
bq.
bq. 1. we try and simply remove Hypersonic SQL catalog period, and then see if the TestDataSourceCatalog passes. If it does, this issue is moot and I will close this ReviewBoard and the issue.
bq.
bq. 2. If #1 doesn't succeed, and suggests changes, we could:
bq. a) flow those changes into DataSourceCatalog -- someone would have to have time to do that :) It also could happen later; or
bq. b) suggest that since this catalog exists and we know it works, temporarily in 0.4 and ongoing until someone refactors or improves DataSourceCatalog ship the HypersonicSQLFriendlyCatalog for folks who want to use it.
bq.
bq. Sound right? Thoughts?
bq.
bq. brian Foster wrote:
bq. i'd be cool with you checking it in for now with a TODO at the top of the class and maybe a JIRA issue to merge the 2 at some point
bq.
bq. Chris Mattmann wrote:
bq. Cool, I will try and use the DataSourceCatalog, see if it passes, and if so, just forget this one. If it doesn't pass, I'll consider the above your +1 and then move forward with this.
bq.
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, here was the issue, check it:
bq.
bq. java.sql.SQLException: Not in aggregate function or group by clause: org.hsqldb.Expression@16ba5c7a in statement [SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p ORDER BY p.product_id DESC ]
bq. at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.executeQuery(Unknown Source)
bq.
bq. Hypersonic SQL doesn't like COUNT functions and ORDER BY to be used together. See: http://www.dcm4che.org/jira/browse/ARR-37
bq.
bq. So, what I did with this catalog is override the getResultListSize, which then caused me to have to override paginateQuery, and then caused me to have to intercept the pagedQuery and then the full paging API, which is what this catalog does, and I think correctly. If I don't hear any further objections, I am going to commit this tomorrow.
bq.
bq. Ricky Nguyen wrote:
bq. Why can't we remove the ORDER clause?
bq.
bq. SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p
bq.
bq. since the order doesn't matter when you're just returning a count.
bq.
bq. Chris Mattmann wrote:
bq. What do you guys think? I'd be OK removing the ORDER clause but it will be a change from the SQL that the DataSourceCatalog used to generate. But, you're right, it doesn't really matter. Wonder why Oracle, and MySQL and Postgres allow it?
bq.
bq. brian Foster wrote:
bq. I vote ditch ORDER!
Hmm VERY interesting. So, I tried to ditch ORDER, and just use DataSourceCatalog. Here's what I got:
[chipotle:oodt/trunk/filemgr] mattmann% more target/surefire-reports/org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog.txt
-------------------------------------------------------------------------------
Test set: org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog
-------------------------------------------------------------------------------
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.828 sec <<< FAILURE!
testFirstProductOnlyOnFirstPage(org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog) Time elapsed: 0.124 sec <<< FAILURE!
junit.framework.ComparisonFailure: expected:<[ShouldBeFirstForPage.tx]t> but was:<[tes]t>
at junit.framework.Assert.assertEquals(Assert.java:81)
at junit.framework.Assert.assertEquals(Assert.java:87)
at org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog.testFirstProductOnlyOnFirstPage(TestDataSourceCatalog.java:278)
[chipotle:oodt/trunk/filemgr] mattmann%
I need to look more, but something is wiggy here.
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
On 2012-03-14 23:40:55, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 23:40:55)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230909#comment-13230909 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-15 00:35:26, brian Foster wrote:
bq. > Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, great question. The reality is that this catalog was hidden away in the File Manager test suite inside of TestDataSourceCatalog. There was something wiggy about the DataSourceCatalog code that Hypersonic SQL didn't like so I had to extend DataSourceCatalog for internal testing since it made sense to use Hypersonic SQL for that (could load up the DBMS in memory, etc.) I know that there was like 1 method that used incompatible SQL syntax that Hypersonic SQL didn't like. Then when I overrode that one method, I had to override like 4-5 others to inject my subclass method into the call chain for that which is what you see in the patch. It seems that DataSourceCatalog has since evolved and it's entirely possible that it would support Hypersonic SQL now, so here's what I see as 2 options:
bq.
bq. 1. we try and simply remove Hypersonic SQL catalog period, and then see if the TestDataSourceCatalog passes. If it does, this issue is moot and I will close this ReviewBoard and the issue.
bq.
bq. 2. If #1 doesn't succeed, and suggests changes, we could:
bq. a) flow those changes into DataSourceCatalog -- someone would have to have time to do that :) It also could happen later; or
bq. b) suggest that since this catalog exists and we know it works, temporarily in 0.4 and ongoing until someone refactors or improves DataSourceCatalog ship the HypersonicSQLFriendlyCatalog for folks who want to use it.
bq.
bq. Sound right? Thoughts?
bq.
bq. brian Foster wrote:
bq. i'd be cool with you checking it in for now with a TODO at the top of the class and maybe a JIRA issue to merge the 2 at some point
bq.
bq. Chris Mattmann wrote:
bq. Cool, I will try and use the DataSourceCatalog, see if it passes, and if so, just forget this one. If it doesn't pass, I'll consider the above your +1 and then move forward with this.
bq.
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, here was the issue, check it:
bq.
bq. java.sql.SQLException: Not in aggregate function or group by clause: org.hsqldb.Expression@16ba5c7a in statement [SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p ORDER BY p.product_id DESC ]
bq. at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.executeQuery(Unknown Source)
bq.
bq. Hypersonic SQL doesn't like COUNT functions and ORDER BY to be used together. See: http://www.dcm4che.org/jira/browse/ARR-37
bq.
bq. So, what I did with this catalog is override the getResultListSize, which then caused me to have to override paginateQuery, and then caused me to have to intercept the pagedQuery and then the full paging API, which is what this catalog does, and I think correctly. If I don't hear any further objections, I am going to commit this tomorrow.
Why can't we remove the ORDER clause?
SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p
since the order doesn't matter when you're just returning a count.
- Ricky
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
On 2012-03-14 23:40:55, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 23:40:55)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232443#comment-13232443 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review6067
-----------------------------------------------------------
Ship it!
LGTM!
- Ricky
On 2012-03-17 01:40:37, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-17 01:40:37)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/DataSourceCatalog.java 1301321
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1301321
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229765#comment-13229765 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/
-----------------------------------------------------------
Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
Summary
-------
DataSourceCatalog compatible with HypersonicSQL
This addresses bug OODT-410.
https://issues.apache.org/jira/browse/OODT-410
Diffs
-----
./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
Diff: https://reviews.apache.org/r/4358/diff
Testing
-------
Thanks,
Chris
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "Chris A. Mattmann (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris A. Mattmann resolved OODT-410.
------------------------------------
Resolution: Fixed
- fixed in r1302489. Thanks for the reviews, all!
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230250#comment-13230250 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-15 00:35:26, brian Foster wrote:
bq. > Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
Hey Brian, great question. The reality is that this catalog was hidden away in the File Manager test suite inside of TestDataSourceCatalog. There was something wiggy about the DataSourceCatalog code that Hypersonic SQL didn't like so I had to extend DataSourceCatalog for internal testing since it made sense to use Hypersonic SQL for that (could load up the DBMS in memory, etc.) I know that there was like 1 method that used incompatible SQL syntax that Hypersonic SQL didn't like. Then when I overrode that one method, I had to override like 4-5 others to inject my subclass method into the call chain for that which is what you see in the patch. It seems that DataSourceCatalog has since evolved and it's entirely possible that it would support Hypersonic SQL now, so here's what I see as 2 options:
1. we try and simply remove Hypersonic SQL catalog period, and then see if the TestDataSourceCatalog passes. If it does, this issue is moot and I will close this ReviewBoard and the issue.
2. If #1 doesn't succeed, and suggests changes, we could:
a) flow those changes into DataSourceCatalog -- someone would have to have time to do that :) It also could happen later; or
b) suggest that since this catalog exists and we know it works, temporarily in 0.4 and ongoing until someone refactors or improves DataSourceCatalog ship the HypersonicSQLFriendlyCatalog for folks who want to use it.
Sound right? Thoughts?
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
On 2012-03-14 23:40:55, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-14 23:40:55)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalog.java PRE-CREATION
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/HsqlDbFriendlyDataSourceCatalogFactory.java PRE-CREATION
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1294690
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231214#comment-13231214 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-15 00:35:26, brian Foster wrote:
bq. > Why can't DataSourceCatalog be extended and just modified per hypersonic needs?... what are the differences exactly between DataSource and Hyersonic versions of the Catalogs?
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, great question. The reality is that this catalog was hidden away in the File Manager test suite inside of TestDataSourceCatalog. There was something wiggy about the DataSourceCatalog code that Hypersonic SQL didn't like so I had to extend DataSourceCatalog for internal testing since it made sense to use Hypersonic SQL for that (could load up the DBMS in memory, etc.) I know that there was like 1 method that used incompatible SQL syntax that Hypersonic SQL didn't like. Then when I overrode that one method, I had to override like 4-5 others to inject my subclass method into the call chain for that which is what you see in the patch. It seems that DataSourceCatalog has since evolved and it's entirely possible that it would support Hypersonic SQL now, so here's what I see as 2 options:
bq.
bq. 1. we try and simply remove Hypersonic SQL catalog period, and then see if the TestDataSourceCatalog passes. If it does, this issue is moot and I will close this ReviewBoard and the issue.
bq.
bq. 2. If #1 doesn't succeed, and suggests changes, we could:
bq. a) flow those changes into DataSourceCatalog -- someone would have to have time to do that :) It also could happen later; or
bq. b) suggest that since this catalog exists and we know it works, temporarily in 0.4 and ongoing until someone refactors or improves DataSourceCatalog ship the HypersonicSQLFriendlyCatalog for folks who want to use it.
bq.
bq. Sound right? Thoughts?
bq.
bq. brian Foster wrote:
bq. i'd be cool with you checking it in for now with a TODO at the top of the class and maybe a JIRA issue to merge the 2 at some point
bq.
bq. Chris Mattmann wrote:
bq. Cool, I will try and use the DataSourceCatalog, see if it passes, and if so, just forget this one. If it doesn't pass, I'll consider the above your +1 and then move forward with this.
bq.
bq.
bq. Chris Mattmann wrote:
bq. Hey Brian, here was the issue, check it:
bq.
bq. java.sql.SQLException: Not in aggregate function or group by clause: org.hsqldb.Expression@16ba5c7a in statement [SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p ORDER BY p.product_id DESC ]
bq. at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
bq. at org.hsqldb.jdbc.jdbcStatement.executeQuery(Unknown Source)
bq.
bq. Hypersonic SQL doesn't like COUNT functions and ORDER BY to be used together. See: http://www.dcm4che.org/jira/browse/ARR-37
bq.
bq. So, what I did with this catalog is override the getResultListSize, which then caused me to have to override paginateQuery, and then caused me to have to intercept the pagedQuery and then the full paging API, which is what this catalog does, and I think correctly. If I don't hear any further objections, I am going to commit this tomorrow.
bq.
bq. Ricky Nguyen wrote:
bq. Why can't we remove the ORDER clause?
bq.
bq. SELECT COUNT(DISTINCT p.product_id) AS numResults FROM GenericFile_metadata p
bq.
bq. since the order doesn't matter when you're just returning a count.
bq.
bq. Chris Mattmann wrote:
bq. What do you guys think? I'd be OK removing the ORDER clause but it will be a change from the SQL that the DataSourceCatalog used to generate. But, you're right, it doesn't really matter. Wonder why Oracle, and MySQL and Postgres allow it?
bq.
bq. brian Foster wrote:
bq. I vote ditch ORDER!
bq.
bq. Chris Mattmann wrote:
bq. Hmm VERY interesting. So, I tried to ditch ORDER, and just use DataSourceCatalog. Here's what I got:
bq.
bq. [chipotle:oodt/trunk/filemgr] mattmann% more target/surefire-reports/org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog.txt
bq. -------------------------------------------------------------------------------
bq. Test set: org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog
bq. -------------------------------------------------------------------------------
bq. Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.828 sec <<< FAILURE!
bq. testFirstProductOnlyOnFirstPage(org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog) Time elapsed: 0.124 sec <<< FAILURE!
bq. junit.framework.ComparisonFailure: expected:<[ShouldBeFirstForPage.tx]t> but was:<[tes]t>
bq. at junit.framework.Assert.assertEquals(Assert.java:81)
bq. at junit.framework.Assert.assertEquals(Assert.java:87)
bq. at org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog.testFirstProductOnlyOnFirstPage(TestDataSourceCatalog.java:278)
bq.
bq. [chipotle:oodt/trunk/filemgr] mattmann%
bq.
bq. I need to look more, but something is wiggy here.
bq.
bq. brian Foster wrote:
bq. When i ditch DataSourceCatalog line number 1793:
bq.
bq. getProductSql += "ORDER BY p.product_id DESC ";
bq.
bq. TestDataSourceCatalog pass for me
bq.
bq. Paul Ramirez wrote:
bq. I'd vote to kick ORDER sorts when not needed; it just adds time to the query execution.
Brian and others FYI my latest patch. The tests still don't pass for me. Here's what I'm running:
[chipotle:oodt/trunk/filemgr] mattmann% mvn -Dtest=TestDataSourceCatalog clean test
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building Catalog and Archive File Management Component
[INFO] task-segment: [clean, test]
[INFO] ------------------------------------------------------------------------
[INFO] [clean:clean {execution: default-clean}]
[INFO] Deleting /Users/mattmann/src/oodt/trunk/fi
....snip....
Failed tests:
testFirstProductOnlyOnFirstPage(org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog)
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.
Please refer to /Users/mattmann/src/oodt/trunk/filemgr/target/surefire-reports for the individual test results.
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11 seconds
[INFO] Finished at: Fri Mar 16 07:15:54 PDT 2012
[INFO] Final Memory: 35M/86M
[INFO] ------------------------------------------------------------------------
[chipotle:oodt/trunk/filemgr] mattmann%
Here is the test output:
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.784 sec <<< FAILURE!
testFirstProductOnlyOnFirstPage(org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog) Time elapsed: 0.122 sec <<< FAILURE!
junit.framework.ComparisonFailure: expected:<[ShouldBeFirstForPage.tx]t> but was:<[tes]t>
at junit.framework.Assert.assertEquals(Assert.java:81)
at junit.framework.Assert.assertEquals(Assert.java:87)
at org.apache.oodt.cas.filemgr.catalog.TestDataSourceCatalog.testFirstProductOnlyOnFirstPage(TestDataSourceCatalog.java:278)
[chipotle:oodt/trunk/filemgr] mattmann%
Ideas?
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review5978
-----------------------------------------------------------
On 2012-03-16 14:14:17, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-16 14:14:17)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/DataSourceCatalog.java 1301321
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1301321
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OODT-410) DataSourceCatalog compatible with
HypersonicSQL
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OODT-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231497#comment-13231497 ]
jiraposter@reviews.apache.org commented on OODT-410:
----------------------------------------------------
bq. On 2012-03-16 16:52:35, Ricky Nguyen wrote:
bq. > ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java, line 769
bq. > <https://reviews.apache.org/r/4358/diff/2/?file=93070#file93070line769>
bq. >
bq. > I think HsqlDbFriendlyDataSourceCatalog.paginateQuery was missing an "ORDER BY ... DESC" clause here, whereas DataSourceCatalog (line 1960) does a descending sort.
bq. >
bq. > Since the primary key product_id is incremented (src/testdata/testcat.sql), and "ShouldBeFirstForPage.txt" is inserted LAST, DataSourceCatalog returns "ShouldBeFirstForPage.txt" as the first item on the FIRST page when ordered DESC.
bq. >
bq. > To me, it seems like DataSourceCatalog works properly and the test case needs to be modified.
I agree... it is a bad test... the test is checking that ShouldBeFirstForPage.txt is the first product for the last page... and in the test there are only 2 pages and the last page has only one product which makes it the last product... whereas the test should be check that ShouldBeFirstForPage.txt is the first product on the first page
- brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4358/#review6043
-----------------------------------------------------------
On 2012-03-16 14:14:17, Chris Mattmann wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4358/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-16 14:14:17)
bq.
bq.
bq. Review request for oodt, brian Foster, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. DataSourceCatalog compatible with HypersonicSQL
bq.
bq.
bq. This addresses bug OODT-410.
bq. https://issues.apache.org/jira/browse/OODT-410
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ./trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/DataSourceCatalog.java 1301321
bq. ./trunk/filemgr/src/test/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1301321
bq.
bq. Diff: https://reviews.apache.org/r/4358/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Chris
bq.
bq.
> DataSourceCatalog compatible with HypersonicSQL
> -----------------------------------------------
>
> Key: OODT-410
> URL: https://issues.apache.org/jira/browse/OODT-410
> Project: OODT
> Issue Type: New Feature
> Components: file manager
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> This patch will provide a customizable extension of the DataSourceCatalog for use with HypersonicSQL DB. All of the existing DataSourceCatalog filemgr.properties config options will be reused.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira