You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by Lewis John Mcgibbney <le...@gmail.com> on 2012/01/15 16:44:54 UTC
Inconsistencies in TestXXXStore
Hi Guys,
Whilst working on the Cassandra tests I've been having a look at
TestHBaseStore [1] and TestSqlStore [2].
>From what I can see the following seems to indicate quite a high degree of
inconsistency between the existing module tests
from TestHBaseStore:
assetSchemaExists
assertPutArray
assertPutBytes
AssertPutMap
testQueryEndKey - currently skipped and has been reported as HBASE-5090
testQueryKeyRange - currently skipped and has been reported as HBASE-5090
testDeleteByQuery - currently skipped and has been reported as HBASE-5090
from TestSqlStore:
testDeleteByQueryFields - currently skipped as we need implement delete
fields in SqlStore
testDeleteByQuery - currently skipped as comments mention HSQLDB hangs for
this test and we need to solve or get another SQLDB for the test suite!
I'm quite confused as to how, or what is exactly going on.
Is it possible for anyone to clarify this please, thank you in advance.
Lewis
[1]
http://svn.apache.org/viewvc/incubator/gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java?view=markup
[2]
http://svn.apache.org/viewvc/incubator/gora/trunk/gora-sql/src/test/java/org/apache/gora/sql/store/TestSqlStore.java?view=markup
--
*Lewis*
Re: Inconsistencies in TestXXXStore
Posted by Ferdy Galema <fe...@kalooga.com>.
I do not have a fully working suggestion yet, but let me get back at
this. It will probably involve the creation of a new base test class,
i.e. AbstractDataStoreTest (just to avoid confusion) and deprecate the
old one. This way datastore tests can gradually be converted.
AbstractDataStoreTest will have some very basic and intuitive methods,
including something like createDataStore(), so overriding datastore test
classes will have to do as little work as possible.
In the mean time if anyone already has some comments/ideas please let
them be heard.
On 01/18/2012 03:13 PM, Lewis John Mcgibbney wrote:
> Hi Ferdy,
>
> Having worked to understand what the tests comprise of I share your view
> that stuff like "assertPutArray" is extremely ambiguous, hence putting up
> large barriers to actually understanding what is going on where and for
> what reasons.
>
> I suppose if you have some idea of how you would like to see the tests
> re-factored then it might be best to get it out in the open as it would
> also help me to visualize how we can progress with this. I'm really keen to
> clean this up as it is going to become a larger and larger problem if and
> when we are able to implement the Accumulo& Solr stores.
>
> On Wed, Jan 18, 2012 at 12:54 PM, Ferdy Galema<fe...@kalooga.com>wrote:
>
>> Hi Lewis,
>>
>> The last time I seriously looked into the Gora tests I was quite surprised
>> by its complexity. That is that all store tests derive from
>> DataStoreTestBase and also set a specific GoraTestDriver. HBase in
>> particular makes it even more difficult by using different frameworks for
>> its tests; TestHBaseStore uses HBaseTestingUtility and the mapreduce tests
>> (which you recently split into two classes) uses the HBaseClusterTestCase
>> class.
>>
>> I am not sure why it is set up the way it is now, but it at the moment it
>> does not suffice. Therefore I think we need to simplify DataStoreTestBase
>> (get rid of GoraTestDriver?) and document all of its methods (certainly the
>> public ones). After DataStoreTestBase is clearly defined, all tests can
>> derive from this class. (If they want to use utility functions than these
>> can simply be put into external utility classes). Only then it makes sense
>> to discuss about differences between datastore test implementations.
>> Because at the moment I simply do not know what for example
>> "assertPutArray" does or should do.
>>
>> Ferdy.
>>
>>
>> On 01/18/2012 12:28 PM, Lewis John Mcgibbney wrote:
>>
>>> Hi Guys,
>>>
>>> Does anyone have comments or thoughts on this one?
>>>
>>> I'm kinda confused...
>>>
>>> Thanks
>>>
>>> On Sun, Jan 15, 2012 at 3:44 PM, Lewis John Mcgibbney<
>>> lewis.mcgibbney@gmail.com> wrote:
>>>
>>> Hi Guys,
>>>> Whilst working on the Cassandra tests I've been having a look at
>>>> TestHBaseStore [1] and TestSqlStore [2].
>>>>
>>>> From what I can see the following seems to indicate quite a high degree
>>>> of
>>>> inconsistency between the existing module tests
>>>>
>>>> from TestHBaseStore:
>>>>
>>>> assetSchemaExists
>>>> assertPutArray
>>>> assertPutBytes
>>>> AssertPutMap
>>>> testQueryEndKey - currently skipped and has been reported as HBASE-5090
>>>> testQueryKeyRange - currently skipped and has been reported as HBASE-5090
>>>> testDeleteByQuery - currently skipped and has been reported as HBASE-5090
>>>>
>>>> from TestSqlStore:
>>>>
>>>> testDeleteByQueryFields - currently skipped as we need implement delete
>>>> fields in SqlStore
>>>> testDeleteByQuery - currently skipped as comments mention HSQLDB hangs
>>>> for
>>>> this test and we need to solve or get another SQLDB for the test suite!
>>>>
>>>> I'm quite confused as to how, or what is exactly going on.
>>>>
>>>> Is it possible for anyone to clarify this please, thank you in advance.
>>>>
>>>> Lewis
>>>>
>>>> [1]
>>>> http://svn.apache.org/viewvc/**incubator/gora/trunk/gora-**
>>>> hbase/src/test/java/org/**apache/gora/hbase/store/**
>>>> TestHBaseStore.java?view=**markup<http://svn.apache.org/viewvc/incubator/gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java?view=markup>
>>>> [2]
>>>> http://svn.apache.org/viewvc/**incubator/gora/trunk/gora-sql/**
>>>> src/test/java/org/apache/gora/**sql/store/TestSqlStore.java?**
>>>> view=markup<http://svn.apache.org/viewvc/incubator/gora/trunk/gora-sql/src/test/java/org/apache/gora/sql/store/TestSqlStore.java?view=markup>
>>>>
>>>> --
>>>> *Lewis*
>>>>
>>>>
>>>>
>
Re: Inconsistencies in TestXXXStore
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi Ferdy,
Having worked to understand what the tests comprise of I share your view
that stuff like "assertPutArray" is extremely ambiguous, hence putting up
large barriers to actually understanding what is going on where and for
what reasons.
I suppose if you have some idea of how you would like to see the tests
re-factored then it might be best to get it out in the open as it would
also help me to visualize how we can progress with this. I'm really keen to
clean this up as it is going to become a larger and larger problem if and
when we are able to implement the Accumulo & Solr stores.
On Wed, Jan 18, 2012 at 12:54 PM, Ferdy Galema <fe...@kalooga.com>wrote:
> Hi Lewis,
>
> The last time I seriously looked into the Gora tests I was quite surprised
> by its complexity. That is that all store tests derive from
> DataStoreTestBase and also set a specific GoraTestDriver. HBase in
> particular makes it even more difficult by using different frameworks for
> its tests; TestHBaseStore uses HBaseTestingUtility and the mapreduce tests
> (which you recently split into two classes) uses the HBaseClusterTestCase
> class.
>
> I am not sure why it is set up the way it is now, but it at the moment it
> does not suffice. Therefore I think we need to simplify DataStoreTestBase
> (get rid of GoraTestDriver?) and document all of its methods (certainly the
> public ones). After DataStoreTestBase is clearly defined, all tests can
> derive from this class. (If they want to use utility functions than these
> can simply be put into external utility classes). Only then it makes sense
> to discuss about differences between datastore test implementations.
> Because at the moment I simply do not know what for example
> "assertPutArray" does or should do.
>
> Ferdy.
>
>
> On 01/18/2012 12:28 PM, Lewis John Mcgibbney wrote:
>
>> Hi Guys,
>>
>> Does anyone have comments or thoughts on this one?
>>
>> I'm kinda confused...
>>
>> Thanks
>>
>> On Sun, Jan 15, 2012 at 3:44 PM, Lewis John Mcgibbney<
>> lewis.mcgibbney@gmail.com> wrote:
>>
>> Hi Guys,
>>>
>>> Whilst working on the Cassandra tests I've been having a look at
>>> TestHBaseStore [1] and TestSqlStore [2].
>>>
>>> From what I can see the following seems to indicate quite a high degree
>>> of
>>> inconsistency between the existing module tests
>>>
>>> from TestHBaseStore:
>>>
>>> assetSchemaExists
>>> assertPutArray
>>> assertPutBytes
>>> AssertPutMap
>>> testQueryEndKey - currently skipped and has been reported as HBASE-5090
>>> testQueryKeyRange - currently skipped and has been reported as HBASE-5090
>>> testDeleteByQuery - currently skipped and has been reported as HBASE-5090
>>>
>>> from TestSqlStore:
>>>
>>> testDeleteByQueryFields - currently skipped as we need implement delete
>>> fields in SqlStore
>>> testDeleteByQuery - currently skipped as comments mention HSQLDB hangs
>>> for
>>> this test and we need to solve or get another SQLDB for the test suite!
>>>
>>> I'm quite confused as to how, or what is exactly going on.
>>>
>>> Is it possible for anyone to clarify this please, thank you in advance.
>>>
>>> Lewis
>>>
>>> [1]
>>> http://svn.apache.org/viewvc/**incubator/gora/trunk/gora-**
>>> hbase/src/test/java/org/**apache/gora/hbase/store/**
>>> TestHBaseStore.java?view=**markup<http://svn.apache.org/viewvc/incubator/gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java?view=markup>
>>> [2]
>>> http://svn.apache.org/viewvc/**incubator/gora/trunk/gora-sql/**
>>> src/test/java/org/apache/gora/**sql/store/TestSqlStore.java?**
>>> view=markup<http://svn.apache.org/viewvc/incubator/gora/trunk/gora-sql/src/test/java/org/apache/gora/sql/store/TestSqlStore.java?view=markup>
>>>
>>> --
>>> *Lewis*
>>>
>>>
>>>
>>
--
*Lewis*
Re: Inconsistencies in TestXXXStore
Posted by Ferdy Galema <fe...@kalooga.com>.
Hi Lewis,
The last time I seriously looked into the Gora tests I was quite
surprised by its complexity. That is that all store tests derive from
DataStoreTestBase and also set a specific GoraTestDriver. HBase in
particular makes it even more difficult by using different frameworks
for its tests; TestHBaseStore uses HBaseTestingUtility and the mapreduce
tests (which you recently split into two classes) uses the
HBaseClusterTestCase class.
I am not sure why it is set up the way it is now, but it at the moment
it does not suffice. Therefore I think we need to simplify
DataStoreTestBase (get rid of GoraTestDriver?) and document all of its
methods (certainly the public ones). After DataStoreTestBase is clearly
defined, all tests can derive from this class. (If they want to use
utility functions than these can simply be put into external utility
classes). Only then it makes sense to discuss about differences between
datastore test implementations. Because at the moment I simply do not
know what for example "assertPutArray" does or should do.
Ferdy.
On 01/18/2012 12:28 PM, Lewis John Mcgibbney wrote:
> Hi Guys,
>
> Does anyone have comments or thoughts on this one?
>
> I'm kinda confused...
>
> Thanks
>
> On Sun, Jan 15, 2012 at 3:44 PM, Lewis John Mcgibbney<
> lewis.mcgibbney@gmail.com> wrote:
>
>> Hi Guys,
>>
>> Whilst working on the Cassandra tests I've been having a look at
>> TestHBaseStore [1] and TestSqlStore [2].
>>
>> From what I can see the following seems to indicate quite a high degree of
>> inconsistency between the existing module tests
>>
>> from TestHBaseStore:
>>
>> assetSchemaExists
>> assertPutArray
>> assertPutBytes
>> AssertPutMap
>> testQueryEndKey - currently skipped and has been reported as HBASE-5090
>> testQueryKeyRange - currently skipped and has been reported as HBASE-5090
>> testDeleteByQuery - currently skipped and has been reported as HBASE-5090
>>
>> from TestSqlStore:
>>
>> testDeleteByQueryFields - currently skipped as we need implement delete
>> fields in SqlStore
>> testDeleteByQuery - currently skipped as comments mention HSQLDB hangs for
>> this test and we need to solve or get another SQLDB for the test suite!
>>
>> I'm quite confused as to how, or what is exactly going on.
>>
>> Is it possible for anyone to clarify this please, thank you in advance.
>>
>> Lewis
>>
>> [1]
>> http://svn.apache.org/viewvc/incubator/gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java?view=markup
>> [2]
>> http://svn.apache.org/viewvc/incubator/gora/trunk/gora-sql/src/test/java/org/apache/gora/sql/store/TestSqlStore.java?view=markup
>>
>> --
>> *Lewis*
>>
>>
>
Re: Inconsistencies in TestXXXStore
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi Guys,
Does anyone have comments or thoughts on this one?
I'm kinda confused...
Thanks
On Sun, Jan 15, 2012 at 3:44 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> Hi Guys,
>
> Whilst working on the Cassandra tests I've been having a look at
> TestHBaseStore [1] and TestSqlStore [2].
>
> From what I can see the following seems to indicate quite a high degree of
> inconsistency between the existing module tests
>
> from TestHBaseStore:
>
> assetSchemaExists
> assertPutArray
> assertPutBytes
> AssertPutMap
> testQueryEndKey - currently skipped and has been reported as HBASE-5090
> testQueryKeyRange - currently skipped and has been reported as HBASE-5090
> testDeleteByQuery - currently skipped and has been reported as HBASE-5090
>
> from TestSqlStore:
>
> testDeleteByQueryFields - currently skipped as we need implement delete
> fields in SqlStore
> testDeleteByQuery - currently skipped as comments mention HSQLDB hangs for
> this test and we need to solve or get another SQLDB for the test suite!
>
> I'm quite confused as to how, or what is exactly going on.
>
> Is it possible for anyone to clarify this please, thank you in advance.
>
> Lewis
>
> [1]
> http://svn.apache.org/viewvc/incubator/gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java?view=markup
> [2]
> http://svn.apache.org/viewvc/incubator/gora/trunk/gora-sql/src/test/java/org/apache/gora/sql/store/TestSqlStore.java?view=markup
>
> --
> *Lewis*
>
>
--
*Lewis*