The Wiki begins..... (was Re: greetings and question about data stores)

[Richard Schilling <rs...@cognitiongroup.biz>]

Craig,

I've finally had a chance to start adding to the Wiki, and I'll be 
putting some thoughts out there tonight.  Any general guidelines?

BTW, I need to talk to you about some recent developments that have come 
up.  I'm going to be renting a space where developers in the Seattle 
area can meet face-to-face and work on projects.  I'll be working with 
the local developers to address the business needs of the open source 
community (in addition to just getting the development done).  I would 
really like you to consider participating, or if there's a person from 
Sun in the Seattle area who can actually show up that would be great too.


Give me a call when you get a chance?

Thanks.

Richard
cell: 206.240.8235


Craig L Russell wrote:
> I've created http://wiki.apache.org/jdo/ScalableBackEnd and linked it 
> from the main JDO wiki.
> 
> Regards,
> 
> Craig
> 
> On Aug 7, 2007, at 10:39 AM, Richard Schilling wrote:
> 
>> Look forward to writing for the Wiki.  I'll go ahead and pull out my 
>> reference material so we can have some time-proven data store texts to 
>> work off of.  That will get us started.
>>
>> Cheers!
>>
>> Richard Schilling
>>
>> Craig L Russell wrote:
>>> Sounds like there's room here for a good discussion, since JDO is the 
>>> only standard datastore-agnostic API that might exploit something 
>>> like MUMPS.
>>> I'll set up a wiki where we can continue the dialog.
>>> Craig
>>> On Aug 4, 2007, at 6:33 PM, Richard Schilling wrote:
>>>> I think an upgrade for FOStore is definitely feasible, but it should 
>>>> be redesigned.  There's no reason I can think of why a robust b-tree 
>>>> type implementation could not be managed.  The Berkeley DB sets are 
>>>> O.K., but they typically work well for small data sets.  Where we 
>>>> have large data sets, performance I think is pretty degraded.
>>>>
>>>> Having said that, I think there's a lot to be learned from an old, 
>>>> but very quick and very robust MUMPS data store.  It's underlying 
>>>> structure is essentially a b-tree, and works very well.  We could 
>>>> take a similar design approach.
>>>>
>>>> Richard Schilling
>>>>
>>>>
>>>> Craig L Russell wrote:
>>>>> Hi Richard,
>>>>> If you're interested in the non-RDBMS side of JDO, I've had for a 
>>>>> long time an interest in upgrading the FOStore piece to use a more 
>>>>> modern key/value store like a modern version of Berkeley DB.
>>>>> The sticking point for many of the non-RDBMS stores in the public 
>>>>> domain is the lack of query capability. And much of this is due to 
>>>>> the lack of schema for the values. In MDR, the key is 
>>>>> not-quite-opaque. You can see this in the special treatment of keys 
>>>>> in the FOStore key-generation. We'd have liked to use a simple long 
>>>>> as a key but MDR doesn't support them. And the value is truly 
>>>>> opaque. In FOStore we simply "serialize" the objects for storage 
>>>>> replacing objects with true FOStore object references.
>>>>> Anyway, if you're interested in upgrading FOStore to use a better 
>>>>> store than MDR, perhaps we can start a wiki to discuss this.
>>>>> Best,
>>>>> Craig
>>>>> On Aug 4, 2007, at 10:46 AM, Richard Schilling wrote:
>>>>>> Looking some more this morning, I also found the source code 
>>>>>> repository for netbeans b-tree implementation:
>>>>>>
>>>>>> http://mdr.netbeans.org/source/browse/mdr/src/org/netbeans/mdr/persistence/btreeimpl/?only_with_tag=release60_m4 
>>>>>>
>>>>>>
>>>>>>
>>>>>> And lo and behold, I also found an old CORBA module I didn't know 
>>>>>> existed.  This is going to get interesting.... :-)
>>>>>>
>>>>>> Richard
>>>>>>
>>>>>>
>>>>>> Craig L Russell wrote:
>>>>>>> Hi Richard,
>>>>>>> Sorry about that. The FOStore implementation had just a bit of 
>>>>>>> documentation which I've tried to locate for you but can't seem 
>>>>>>> to find just now. When I find it I'll post it on the JDO site and 
>>>>>>> put a link here.
>>>>>>> Craig
>>>>>>> On Aug 3, 2007, at 11:07 PM, Richard Schilling wrote:
>>>>>>>> Here is the posting I had way back in March of 2006, which 
>>>>>>>> didn't get much response - if any at all:
>>>>>>>>
>>>>>>>> http://www.webservertalk.com/archive383-2006-3-1437593.html
>>>>>>>>
>>>>>>>> Never did find out the answers to these questions, but this 
>>>>>>>> explains what I'm up to.
>>>>>>>>
>>>>>>>> After just looking, I find this on the netbeans site:
>>>>>>>>
>>>>>>>> http://mdr.netbeans.org/architecture.html#persistence
>>>>>>>>
>>>>>>>> This explains MDR, the use of b-tree (which the JDO reference 
>>>>>>>> implementation uses) in much more detail than I saw in the past.
>>>>>>>>
>>>>>>>> FYI.
>>>>>>>>
>>>>>>>> Cheers!
>>>>>>>>
>>>>>>>> Richard Schilling
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Craig L Russell wrote:
>>>>>>>>> Hi Richard,
>>>>>>>>> The fostore project at Apache JDO is an implementation that you 
>>>>>>>>> could study for insights as to how the various components 
>>>>>>>>> interact. There's a separation of an abstract StoreManager 
>>>>>>>>> responsible for storing data to a specific datastore, a 
>>>>>>>>> StateManager responsible for managing the state of individual 
>>>>>>>>> instances, and a PersistenceManager whose API is pretty much 
>>>>>>>>> defined in the JDO specification.
>>>>>>>>> Regards,
>>>>>>>>> Craig
>>>>>>>>> On Jul 31, 2007, at 3:23 PM, Richard Schilling wrote:
>>>>>>>>>> I did realize that the JDO API and the model in the apache 
>>>>>>>>>> code was just the implementation of the spec.  What I'm after 
>>>>>>>>>> is to make sure I understand what parts of the API (which 
>>>>>>>>>> interfaces) are used by a specific implementation to cause 
>>>>>>>>>> classes to be committed to the data store.  It looks like the 
>>>>>>>>>> transactions defined in the spec serve this purpose, but 
>>>>>>>>>> there's so much information I can't be sure.
>>>>>>>>>>
>>>>>>>>>> Thanks ... still getting my head around the JDO spec - I've 
>>>>>>>>>> even read the JDO book which I have.
>>>>>>>>>>
>>>>>>>>>> Cheers.
>>>>>>>>>> Richard
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Matthew T. Adams wrote:
>>>>>>>>>>> Hi Richard,
>>>>>>>>>>> Apache JDO is the home of the JDO API (the interface & class 
>>>>>>>>>>> files of the
>>>>>>>>>>> specification) & TCK (the code that tests whether a JDO 
>>>>>>>>>>> implementation is
>>>>>>>>>>> compliant with the specification); none of the code here 
>>>>>>>>>>> actually writes to
>>>>>>>>>>> a database*.  The reference implementation for JDO 2.0 is JPOX
>>>>>>>>>>> (www.jpox.org) -- it is an implementation of the JDO 2.0 
>>>>>>>>>>> API.  There are
>>>>>>>>>>> also many other implementations of JDO out there.
>>>>>>>>>>> HTH,
>>>>>>>>>>> Matthew
>>>>>>>>>>> *:  There is a legacy reference implementation called FOStore 
>>>>>>>>>>> (pronounced
>>>>>>>>>>> like "foster") that was used as the JDO 1.0, but I don't 
>>>>>>>>>>> think that's used
>>>>>>>>>>> much anymore.  Others can comment on that.
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Richard Schilling 
>>>>>>>>>>> [mailto:rschilling@cognitiongroup.biz] Sent: Tuesday, July 
>>>>>>>>>>> 31, 2007 10:22 AM
>>>>>>>>>>> To: jdo-dev@db.apache.org
>>>>>>>>>>> Subject: greetings and question about data stores
>>>>>>>>>>> Greetings.  This is my first post to the list.
>>>>>>>>>>> I would like to know about the code base itself.  There is 
>>>>>>>>>>> not any documentation that discusses the structure of the 
>>>>>>>>>>> code itself and how the software interacts with the data store.
>>>>>>>>>>> Can any one tell me what source code files actually contain 
>>>>>>>>>>> the code that writes to the data store?
>>>>>>>>>>> Thanks!
>>>>>>>>>>> Richard Schilling
>>>>>>>>>>
>>>>>>>>> Craig Russell
>>>>>>>>> Architect, Sun Java Enterprise System 
>>>>>>>>> http://java.sun.com/products/jdo
>>>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>>>
>>>>>>> Craig Russell
>>>>>>> Architect, Sun Java Enterprise System 
>>>>>>> http://java.sun.com/products/jdo
>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>
>>>>> Craig Russell
>>>>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>> P.S. A good JDO? O, Gasp!
>>>>
>>> Craig Russell
>>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>> P.S. A good JDO? O, Gasp!
>>
> 
> Craig Russell
> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
> 408 276-5638 mailto:Craig.Russell@sun.com
> P.S. A good JDO? O, Gasp!
>