You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by "Keith Turner (Created) (JIRA)" <ji...@apache.org> on 2011/12/10 01:34:40 UTC
[jira] [Created] (GORA-66) testDeleteByQueryFields seems incorrect
testDeleteByQueryFields seems incorrect
---------------------------------------
Key: GORA-66
URL: https://issues.apache.org/jira/browse/GORA-66
Project: Gora
Issue Type: Bug
Reporter: Keith Turner
While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
* The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
* The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-66) testDeleteByQueryFields seems incorrect
Posted by "Lewis John McGibbney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated GORA-66:
-------------------------------------
Affects Version/s: 0.2
Fix Version/s: 0.3
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Apache Gora
> Issue Type: Bug
> Affects Versions: 0.2
> Reporter: Keith Turner
> Fix For: 0.3
>
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-66) testDeleteByQueryFields seems
incorrect
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168506#comment-13168506 ]
Lewis John McGibbney commented on GORA-66:
------------------------------------------
Well currently this particular test fails for both the gora-hbase [1] and gora-sql [2] modules anyway, however it doesn't seem to be for the reasons you highlight!
I suppose first we need to determine whether this test should flush after the particular call FOR ALL STORES.
[1] https://builds.apache.org/job/gora-trunk/org.apache.gora$gora-hbase/67/testReport/org.apache.gora.hbase.store/TestHBaseStore/testDeleteByQueryFields/
[2] https://builds.apache.org/job/gora-trunk/org.apache.gora$gora-sql/67/testReport/org.apache.gora.sql.store/TestSqlStore/testDeleteByQueryFields/
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Gora
> Issue Type: Bug
> Reporter: Keith Turner
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-66) testDeleteByQueryFields seems
incorrect
Posted by "Keith Turner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168497#comment-13168497 ]
Keith Turner commented on GORA-66:
----------------------------------
The patch is kinda of a question. I think the changes may need to be made, but I am looking for guidance. I did not try running the change against the other backends, the existing unit test seem to hang or fail. Not sure if this is because I am running on a Mac. I will try to run them on a linux box.
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Gora
> Issue Type: Bug
> Reporter: Keith Turner
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-66) testDeleteByQueryFields seems incorrect
Posted by "Keith Turner (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keith Turner updated GORA-66:
-----------------------------
Attachment: GORA-66-1.patch
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Gora
> Issue Type: Bug
> Reporter: Keith Turner
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (GORA-66) testDeleteByQueryFields
seems incorrect
Posted by "Lewis John McGibbney (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182779#comment-13182779 ]
Lewis John McGibbney edited comment on GORA-66 at 1/9/12 8:41 PM:
------------------------------------------------------------------
OK having been on the HBase user@ list to try to clarify how HBase treats end keys I can confirm that the test should treat them as exclusive. This is indicated in the link below
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html#setStopRow(byte[])
So it appears you are correct Keith, however this still seems to break the build... some debugging is required I think.
was (Author: lewismc):
OK having been on the HBase user@ list to try to clarify how HBase treats end keys I can confirm that the test should treat them as exclusive. This is indicated in the link below
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Gora
> Issue Type: Bug
> Reporter: Keith Turner
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-66) testDeleteByQueryFields seems
incorrect
Posted by "Keith Turner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183353#comment-13183353 ]
Keith Turner commented on GORA-66:
----------------------------------
Are you asking about changing the '>=' to a '>' ?
I think the test was checking that everything greater than or equal to the end key was not deleted. However if the end key is inclusive, then this check is wrong. It should ensure that everything greater than the end key was not deleted. I think the the following should be true after a delete by query, this is the change in the patch.
Every row r such that r >= startKey AND r <= endKey was deleted.
Every row r such that r < startKey OR r > endKey was not deleted.
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Gora
> Issue Type: Bug
> Reporter: Keith Turner
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-66) testDeleteByQueryFields seems
incorrect
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182779#comment-13182779 ]
Lewis John McGibbney commented on GORA-66:
------------------------------------------
OK having been on the HBase user@ list to try to clarify how HBase treats end keys I can confirm that the test should treat them as exclusive. This is indicated in the link below
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Gora
> Issue Type: Bug
> Reporter: Keith Turner
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-66) testDeleteByQueryFields seems
incorrect
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182625#comment-13182625 ]
Lewis John McGibbney commented on GORA-66:
------------------------------------------
After applying and committing this breaks HBase test implementation. I will keep the patch implemented for the time being. I suspect that its the inclusive nature of the end key that is making the test fail.
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Gora
> Issue Type: Bug
> Reporter: Keith Turner
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-66) testDeleteByQueryFields seems
incorrect
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182570#comment-13182570 ]
Hudson commented on GORA-66:
----------------------------
Integrated in gora-trunk #98 (See [https://builds.apache.org/job/gora-trunk/98/])
commit to address GORA-66 and update to CHANGES.txt. Thank you Keith for the patch
lewismc : http://svn.apache.org/viewvc/incubator/gora/trunk/viewvc/?view=rev&root=&revision=1229178
Files :
* /incubator/gora/trunk/CHANGES.txt
* /incubator/gora/trunk/gora-core/src/test/java/org/apache/gora/store/DataStoreTestUtil.java
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Gora
> Issue Type: Bug
> Reporter: Keith Turner
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-66) testDeleteByQueryFields seems
incorrect
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182784#comment-13182784 ]
Lewis John McGibbney commented on GORA-66:
------------------------------------------
Having written that then re-reading it I'm not entirely sure this is correct.
{bq} As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive. {bq}
The patch treats the end key as exclusive now rather than inclusive before is this correct? HBase treats as exclusive, we need to revert this part of the patch... Yes?
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Gora
> Issue Type: Bug
> Reporter: Keith Turner
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-66) testDeleteByQueryFields seems
incorrect
Posted by "Enis Soztutar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182396#comment-13182396 ]
Enis Soztutar commented on GORA-66:
-----------------------------------
bq. I suppose first we need to determine whether this test should flush after the particular call FOR ALL STORES.
Yes, the test should call flush() for all stores. The semantics for delete, is like the semantics for put(), in that, you have to call flush() or close() to make sure that the changes are committed.
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Gora
> Issue Type: Bug
> Reporter: Keith Turner
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-66) testDeleteByQueryFields seems
incorrect
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167105#comment-13167105 ]
Lewis John McGibbney commented on GORA-66:
------------------------------------------
Hi Keith, I applied your patch and have been looking at my logs after testing the entire suite. I wonder if you are aware if the patch has any affect on the existing tests, before it were to be used on your Accumulo backend? In the interim, once I can understand this I would be happy to work with you to get thus integrated into the code base. Thank you
> testDeleteByQueryFields seems incorrect
> ---------------------------------------
>
> Key: GORA-66
> URL: https://issues.apache.org/jira/browse/GORA-66
> Project: Gora
> Issue Type: Bug
> Reporter: Keith Turner
> Attachments: GORA-66-1.patch
>
>
> While writing the Accumulo backend I have been trying to run some of the existing test against it. It seems that the test DataStoreTestUtil.testDeleteByQueryFields() has two problems.
> * The test does not flush the store after calling deleteByQuery(). This is needed for accumulo since deletes are just inserts and must be flushed.
> * The test expects the last key in a range that was deleted to be present. As far as I can tell the end key in a query should be inclusive, but the test treats as exclusive.
> I will submit a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira