You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Spencer Gibb <sp...@livremed.com> on 2009/03/07 01:46:29 UTC
Google App Engine DataStore API on HBase in java
In an interview with InfoQ[1] the hbase project leads said:
"...an implementation of the Google App Engine DataStore API that went
against HBase and that parsed GQL, etc., is a contribution we wouldn't
say no to."
I've been looking at doing a project using hbase (it seems the closest
to bigtable I can find). I've also done some work on google appengine
and really like their datastore. I've looked at some of the ORM'ish
things linked from the wiki but found none of them came close to
datastore.
With that said, I've started implementing the Google App Engine
DataStore API in java. You can find the source here:
http://code.google.com/p/datastore/source/browse/
It is rough at this point.
Current capabilities:
- get/put/delete Model objects.
- query for all of one kind of Model
- parse simple gql (SELECT * FROM Person LIMIT 20, 100)
Future capabilities:
- be as close as possible (within java's limits) to the datasore api [2]
- be compatible with index.yaml [3]
- have different implementations (such as hypertable)
Differences:
Python's dynamic nature means there will be differences with a java
api. Here are some of them.
-overriding static methods
python: Person.all()
java: Datastore.all(Person.class)
-keyword arguments
python: s = Story(title="The Three Little Pigs")
java: Store s = new Story(); s.setTitle("...")
python: Story.get_by_id(123, parent=parentStory)
java: Datastore.getById(parentStory, 123);
-dynamic properties
python: name = db.StringProperty(required=True)
java: @StringProperty(required=true) String name;
And probably more I can't think of right now.
Anyway, browse the code, let me know what you think.
--
spencer@livremed.com
References:
[1] http://www.infoq.com/news/2008/04/hbase-interview
[2] http://code.google.com/appengine/docs/python/datastore/
[3] http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html#Defining_Indexes_With_index_yaml
Re: Google App Engine DataStore API on HBase in java
Posted by Spencer Gibb <sp...@livremed.com>.
I'm not sure if anything is missing. The basic premise is that there
is one table for entities and various others for indexes (according to
the datastore team lead [1]).
The hbase jars are in the datastore svn on googlecode. [2]
Parsing gql is really simple with javacc [3]. Making that gql do
something, that's another thing.
I'll definitely look at openengine. According to his tag line he's
going for the whole app-engine, but he only has code for datastore.
Thanks,
--
Spencer
References:
[1] https://sites.google.com/site/io/under-the-covers-of-the-google-app-engine-datastore
[2] http://datastore.googlecode.com/svn/maven2/
[3] https://javacc.dev.java.net/
On Sat, Mar 7, 2009 at 2:31 PM, stack <st...@duboce.net> wrote:
> Excellent!
>
> What is missing from hbase that you need to support your app engine work?
>
> On the differences from python, I think you are going about it the right way
> where you keep a running list that implementers used-to pythonisms can
> consult.
>
> I'd suggest you add a link to your project to the "Supporting Projects"
> page.
>
> Are the hbase jars now available in a maven repo? (It looks like it reading
> your pom).
>
> Are you writing your own gql parser? (It looks that way -- thats great).
>
> You might want to take a look at Michael Gottesman's skunkworks openengine
> project over on github: http://github.com/gottesmm/openengine/tree/master
>
> St.Ack
>
> On Fri, Mar 6, 2009 at 4:46 PM, Spencer Gibb <sp...@livremed.com> wrote:
>
>> In an interview with InfoQ[1] the hbase project leads said:
>>
>> "...an implementation of the Google App Engine DataStore API that went
>> against HBase and that parsed GQL, etc., is a contribution we wouldn't
>> say no to."
>>
>> I've been looking at doing a project using hbase (it seems the closest
>> to bigtable I can find). I've also done some work on google appengine
>> and really like their datastore. I've looked at some of the ORM'ish
>> things linked from the wiki but found none of them came close to
>> datastore.
>>
>> With that said, I've started implementing the Google App Engine
>> DataStore API in java. You can find the source here:
>>
>> http://code.google.com/p/datastore/source/browse/
>>
>> It is rough at this point.
>>
>> Current capabilities:
>> - get/put/delete Model objects.
>> - query for all of one kind of Model
>> - parse simple gql (SELECT * FROM Person LIMIT 20, 100)
>>
>> Future capabilities:
>> - be as close as possible (within java's limits) to the datasore api [2]
>> - be compatible with index.yaml [3]
>> - have different implementations (such as hypertable)
>>
>> Differences:
>> Python's dynamic nature means there will be differences with a java
>> api. Here are some of them.
>> -overriding static methods
>> python: Person.all()
>> java: Datastore.all(Person.class)
>>
>> -keyword arguments
>> python: s = Story(title="The Three Little Pigs")
>> java: Store s = new Story(); s.setTitle("...")
>> python: Story.get_by_id(123, parent=parentStory)
>> java: Datastore.getById(parentStory, 123);
>>
>> -dynamic properties
>> python: name = db.StringProperty(required=True)
>> java: @StringProperty(required=true) String name;
>>
>> And probably more I can't think of right now.
>>
>> Anyway, browse the code, let me know what you think.
>>
>> --
>> spencer@livremed.com
>>
>> References:
>> [1] http://www.infoq.com/news/2008/04/hbase-interview
>> [2] http://code.google.com/appengine/docs/python/datastore/
>> [3]
>> http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html#Defining_Indexes_With_index_yaml
>>
>
Re: Google App Engine DataStore API on HBase in java
Posted by stack <st...@duboce.net>.
Excellent!
What is missing from hbase that you need to support your app engine work?
On the differences from python, I think you are going about it the right way
where you keep a running list that implementers used-to pythonisms can
consult.
I'd suggest you add a link to your project to the "Supporting Projects"
page.
Are the hbase jars now available in a maven repo? (It looks like it reading
your pom).
Are you writing your own gql parser? (It looks that way -- thats great).
You might want to take a look at Michael Gottesman's skunkworks openengine
project over on github: http://github.com/gottesmm/openengine/tree/master
St.Ack
On Fri, Mar 6, 2009 at 4:46 PM, Spencer Gibb <sp...@livremed.com> wrote:
> In an interview with InfoQ[1] the hbase project leads said:
>
> "...an implementation of the Google App Engine DataStore API that went
> against HBase and that parsed GQL, etc., is a contribution we wouldn't
> say no to."
>
> I've been looking at doing a project using hbase (it seems the closest
> to bigtable I can find). I've also done some work on google appengine
> and really like their datastore. I've looked at some of the ORM'ish
> things linked from the wiki but found none of them came close to
> datastore.
>
> With that said, I've started implementing the Google App Engine
> DataStore API in java. You can find the source here:
>
> http://code.google.com/p/datastore/source/browse/
>
> It is rough at this point.
>
> Current capabilities:
> - get/put/delete Model objects.
> - query for all of one kind of Model
> - parse simple gql (SELECT * FROM Person LIMIT 20, 100)
>
> Future capabilities:
> - be as close as possible (within java's limits) to the datasore api [2]
> - be compatible with index.yaml [3]
> - have different implementations (such as hypertable)
>
> Differences:
> Python's dynamic nature means there will be differences with a java
> api. Here are some of them.
> -overriding static methods
> python: Person.all()
> java: Datastore.all(Person.class)
>
> -keyword arguments
> python: s = Story(title="The Three Little Pigs")
> java: Store s = new Story(); s.setTitle("...")
> python: Story.get_by_id(123, parent=parentStory)
> java: Datastore.getById(parentStory, 123);
>
> -dynamic properties
> python: name = db.StringProperty(required=True)
> java: @StringProperty(required=true) String name;
>
> And probably more I can't think of right now.
>
> Anyway, browse the code, let me know what you think.
>
> --
> spencer@livremed.com
>
> References:
> [1] http://www.infoq.com/news/2008/04/hbase-interview
> [2] http://code.google.com/appengine/docs/python/datastore/
> [3]
> http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html#Defining_Indexes_With_index_yaml
>
Re: Google App Engine DataStore API on HBase in java
Posted by Spencer Gibb <sp...@livremed.com>.
I got alot of information from this talk.
https://sites.google.com/site/io/under-the-covers-of-the-google-app-engine-datastore
On Fri, Mar 6, 2009 at 5:46 PM, Spencer Gibb <sp...@livremed.com> wrote:
> In an interview with InfoQ[1] the hbase project leads said:
>
> "...an implementation of the Google App Engine DataStore API that went
> against HBase and that parsed GQL, etc., is a contribution we wouldn't
> say no to."
>
> I've been looking at doing a project using hbase (it seems the closest
> to bigtable I can find). I've also done some work on google appengine
> and really like their datastore. I've looked at some of the ORM'ish
> things linked from the wiki but found none of them came close to
> datastore.
>
> With that said, I've started implementing the Google App Engine
> DataStore API in java. You can find the source here:
>
> http://code.google.com/p/datastore/source/browse/
>
> It is rough at this point.
>
> Current capabilities:
> - get/put/delete Model objects.
> - query for all of one kind of Model
> - parse simple gql (SELECT * FROM Person LIMIT 20, 100)
>
> Future capabilities:
> - be as close as possible (within java's limits) to the datasore api [2]
> - be compatible with index.yaml [3]
> - have different implementations (such as hypertable)
>
> Differences:
> Python's dynamic nature means there will be differences with a java
> api. Here are some of them.
> -overriding static methods
> python: Person.all()
> java: Datastore.all(Person.class)
>
> -keyword arguments
> python: s = Story(title="The Three Little Pigs")
> java: Store s = new Story(); s.setTitle("...")
> python: Story.get_by_id(123, parent=parentStory)
> java: Datastore.getById(parentStory, 123);
>
> -dynamic properties
> python: name = db.StringProperty(required=True)
> java: @StringProperty(required=true) String name;
>
> And probably more I can't think of right now.
>
> Anyway, browse the code, let me know what you think.
>
> --
> spencer@livremed.com
>
> References:
> [1] http://www.infoq.com/news/2008/04/hbase-interview
> [2] http://code.google.com/appengine/docs/python/datastore/
> [3] http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html#Defining_Indexes_With_index_yaml
>