You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by Lewis John Mcgibbney <le...@gmail.com> on 2012/03/10 02:33:10 UTC
Obtaining Datastore Mapping's during testing time.
Hi,
I'm back looking at the Cassandra tests again and I've nearly cracked them.
What I don't understand is how to obtain the mapping's from
conf/gora-cassandra-mapping.xml?
What I have so far is
lewis@lewis-01:~/ASF/gora/gora-cassandra/src/test$ tree
.
├── conf
│ ├── cassandra.yaml
│ ├── gora-cassandra-mapping.xml
│ ├── gora.properties
└── java
└── org
└── apache
└── gora
└── cassandra
├── GoraCassandraTestDriver.java
└── store
└── TestCassandraStore.java
Looking at the TestHbaseStore I see that we call
@Override
public void setUp() throws Exception {
super.setUp();
conf = getTestDriver().getHbaseUtil().getConfiguration();
}
Is this what is required to obtain the mapping configuration? This is easy
using HBaseTestingUtility as it's a method in the class, unfortunately
neither CassandraDaemon (Cassandra API) nor EmbeddedServerHelper (Hector
API) have this method to simply call on...
When I try to run my tests my Cassandra configuration is fine, but no
mapping seems to be retrieved and most of the test log output complains
testTruncateSchema(org.apache.gora.cassandra.store.TestCassandraStore):
org.apache.avro.generic.GenericArray.add(Ljava/lang/Object;)V
testTruncateSchema(org.apache.gora.cassandra.store.TestCassandraStore):
InvalidRequestException(why:Keyspace does not exist.)
This is because I'm not picking up the mappings from conf. I've uploaded my
latest patch to Jira @ GORA-53-v8.patch ha ha ha!!! I would seriously
appreciate if someone could shine some light on it for me/put me out my
misery/stop me from going insane in the membrane... etc.
This is driving me mad.
Thanks
Lewis
Re: Obtaining Datastore Mapping's during testing time.
Posted by Lewis John Mcgibbney <le...@gmail.com>.
I've finally understood what is still going wrong here (hopefully). If you
have a look at TestHBaseStore#createEmployeeDataStore [0],
#createWebPageDataStore [0],
@Override
protected DataStore<String, Employee> createEmployeeDataStore()
throws IOException {
return DataStoreFactory.createDataStore(HBaseStore.class, String.class,
Employee.class, conf);
}
@Override
protected DataStore<String, WebPage> createWebPageDataStore()
throws IOException {
return DataStoreFactory.createDataStore(HBaseStore.class, String.class,
WebPage.class, conf);
}
and TestSqlStore#createEmployeeDataStore [1] and #createWebPageDataStore
[1]
@Override
protected DataStore<String, Employee> createEmployeeDataStore()
throws IOException {
SqlStore<String, Employee> store = new SqlStore<String, Employee>();
store.initialize(String.class, Employee.class, DataStoreFactory.properties);
return store;
}
@Override
protected DataStore<String, WebPage> createWebPageDataStore() throws
IOException {
SqlStore<String, WebPage> store = new SqlStore<String, WebPage>();
store.initialize(String.class, WebPage.class, DataStoreFactory.properties);
return store;
}
they are being constructed in different ways. Right now although I have the
embedded Cassandra cluster being initiated, I'm getting lengthy stacktraces
indicating the the configuration is not correct when attempting to create
the appopriate DataStore (in this case CassandraStore).
If anyone has any suggestions about how I should be constructing the above
TestCassandraStore#createEmployeeDataStore classes then I would be really
happy to hear it... until then it's back to the drawing board I guess.
[0]
http://svn.apache.org/repos/asf/gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java
[1]
http://svn.apache.org/repos/asf/gora/trunk/gora-sql/src/test/java/org/apache/gora/sql/store/TestSqlStore.java
On Sat, Mar 10, 2012 at 12:10 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> Hi Enis
>
> On Sat, Mar 10, 2012 at 2:04 AM, Enis Söztutar <en...@apache.org> wrote:
>
>> Did you tried putting the test conf under gora-cassandra/src/test/conf/ ?
>> See the files in gora-hbase/src/test/conf/.
>>
>> Yeah I mean my test mappings and the cassandra.yaml (equivalent of HBase
> hbase-site.xml) is underneath the conf directory (actually I'm going to
> remove gora.properties from this folder as it currently does not play any
> role). I just seem to be having a problem actually obtaining the mappings
> from anywhere, and also I currently don't know/can't find the correct code
> to initiate the Hadoop Configuration. I'm going to attach my most recent
> sure-fire report to the Jira issue as well so people can see the output.
>
> The classpath and test classpath is cofigured in build-common.xml:
>>
>
> Does maven pick these properties up? I thought this was for Ant only? I
> have not used Ant with Gora for a while now, and I'm aware that the Ant
> build is broken and as far as I'm aware unmaintained now.
>
> Thanks Enis
>
> Lewis
>
--
*Lewis*
Re: Obtaining Datastore Mapping's during testing time.
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi Enis
On Sat, Mar 10, 2012 at 2:04 AM, Enis Söztutar <en...@apache.org> wrote:
> Did you tried putting the test conf under gora-cassandra/src/test/conf/ ?
> See the files in gora-hbase/src/test/conf/.
>
> Yeah I mean my test mappings and the cassandra.yaml (equivalent of HBase
hbase-site.xml) is underneath the conf directory (actually I'm going to
remove gora.properties from this folder as it currently does not play any
role). I just seem to be having a problem actually obtaining the mappings
from anywhere, and also I currently don't know/can't find the correct code
to initiate the Hadoop Configuration. I'm going to attach my most recent
sure-fire report to the Jira issue as well so people can see the output.
The classpath and test classpath is cofigured in build-common.xml:
>
Does maven pick these properties up? I thought this was for Ant only? I
have not used Ant with Gora for a while now, and I'm aware that the Ant
build is broken and as far as I'm aware unmaintained now.
Thanks Enis
Lewis
Re: Obtaining Datastore Mapping's during testing time.
Posted by Enis Söztutar <en...@apache.org>.
Hi Lewis,
Please don't go mad. we need you : )
Did you tried putting the test conf under gora-cassandra/src/test/conf/ ?
See the files in gora-hbase/src/test/conf/.
The classpath and test classpath is cofigured in build-common.xml:
<path id="classpath">
<pathelement location="${build.classes.dir}"/>
<pathelement location="${conf.dir}"/>
<fileset dir="${lib.dir}">
<include name="*.jar" />
</fileset>
</path>
<!-- the unit test classpath: uses test.src.dir for configuration -->
<path id="test.classpath">
<pathelement location="${examples.build.classes.dir}"/>
<pathelement location="${conf.dir}"/>
<pathelement location="${test.build.classes.dir}" />
<pathelement location="${test.src.dir}"/>
<pathelement location="${build.dir}"/>
<pathelement location="${test.conf.dir}"/>
<path refid="classpath"/>
</path>
Enis
On Fri, Mar 9, 2012 at 5:33 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> Hi,
>
> I'm back looking at the Cassandra tests again and I've nearly cracked them.
>
> What I don't understand is how to obtain the mapping's from
> conf/gora-cassandra-mapping.xml?
>
> What I have so far is
>
> lewis@lewis-01:~/ASF/gora/gora-cassandra/src/test$ tree
> .
> ├── conf
> │ ├── cassandra.yaml
> │ ├── gora-cassandra-mapping.xml
> │ ├── gora.properties
> └── java
> └── org
> └── apache
> └── gora
> └── cassandra
> ├── GoraCassandraTestDriver.java
> └── store
> └── TestCassandraStore.java
>
>
> Looking at the TestHbaseStore I see that we call
>
> @Override
> public void setUp() throws Exception {
> super.setUp();
> conf = getTestDriver().getHbaseUtil().getConfiguration();
> }
>
> Is this what is required to obtain the mapping configuration? This is easy
> using HBaseTestingUtility as it's a method in the class, unfortunately
> neither CassandraDaemon (Cassandra API) nor EmbeddedServerHelper (Hector
> API) have this method to simply call on...
>
> When I try to run my tests my Cassandra configuration is fine, but no
> mapping seems to be retrieved and most of the test log output complains
>
> testTruncateSchema(org.apache.gora.cassandra.store.TestCassandraStore):
> org.apache.avro.generic.GenericArray.add(Ljava/lang/Object;)V
> testTruncateSchema(org.apache.gora.cassandra.store.TestCassandraStore):
> InvalidRequestException(why:Keyspace does not exist.)
>
> This is because I'm not picking up the mappings from conf. I've uploaded my
> latest patch to Jira @ GORA-53-v8.patch ha ha ha!!! I would seriously
> appreciate if someone could shine some light on it for me/put me out my
> misery/stop me from going insane in the membrane... etc.
>
> This is driving me mad.
>
> Thanks
>
> Lewis
>